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Abstract 

In  modern  warfare,  Tactical  Unmanned  Aerial  Vehicles  (TUAVs)  are  rapidly 
taking  on  a  leading  role  in  traditional  and  non-traditional  ISR,  to  include  Automatic 
Target  Recognition  (ATR).  However,  additional  advancements  in  processors  and  sensors 
on  TUAVs  are  still  needed  before  they  can  be  widely  employed  as  a  primary  source  for 
positive  identification  in  the  Combat  Identification  (CID)  process.  Cost  is  a  driving 
factor  for  operating  an  ATR  system  using  multiple  TUAVs.  The  cost  of  high  quality 
sensors  appropriate  for  a  single  TUAV  can  be  significantly  higher  than  less  sophisticated 
sensors  suitable  for  deployment  on  a  group,  or  swarm,  of  coordinated  TUAVs. 

Employing  two  or  more  coordinated  TUAVs  with  less  complex  sensors  may  lead  to  an 
equivalent  or  even  better  CID  call  than  sending  a  single  TUAV  with  more  sophisticated 
sensors  at  a  significantly  higher  cost.  In  addition,  the  coordinated  TUAVs  may  be  capable 
of  reducing  the  time  needed  to  correctly  discriminate  an  object. 

In  our  study  we  construct  a  simulation  model  of  a  single  TUAV  system  with  a 
high  quality  sensor  and  competing  TUAV  swarm  systems  using  less  capable  sensors  in 
Arena.  We  use  the  Boolean  OR  rule  in  our  fusion  algorithm  to  combine  the  declaration 
of  sensors  modeled  in  the  swarm  system.  Five  measures  of  performance  (accuracy, 
number  of  TUAVs  shot  down,  TUAV  preparation  time,  mean  of  decision  time,  mean  of 
simulated  mission  time)  from  the  simulation  models  are  collected  to  compare  the  swarm 
system  to  the  single  TUAV  system.  Statistical  comparisons  are  conducted  using  a  paired 
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t-test.  The  results  illustrate  improved  performance  of  our  swarm  systems  across  most 
measures  of  performance. 


v 


AFIT/GOR/ENS/08-09 


Acknowledgments 


I  would  like  to  express  my  sincere  gratitude  to  my  faculty  advisor,  Dr.  J.O.  Miller, 
for  his  guidance,  correction,  and  support  throughout  the  course  of  this  thesis  effort.  The 
insight  and  experience  was  certainly  appreciated.  I  would,  also,  like  to  thank  my  sponsor, 
Capt.  Carey  Tucker,  from  the  Air  Force  Materiel  Command  for  both  the  support  and 
latitude  provided  to  me  in  this  endeavor.  I  am,  also,  indebted  to  ROK  Army  which 
provided  me  this  education  opportunity  and  many  supports 

Finally,  I  would  like  to  thank  my  lovely  wife  who  always  inspires  me  to  further 

efforts. 


Younin  Kang 


Table  of  Contents 


Page 

Abstract  . iv 

Acknowledgments . vi 

List  of  Figures . ix 

List  of  Tables . xi 

I.  Introduction . 1 

Background . 1 

Research  Problem . 3 

Scope . 4 

Thesis  Organization . 4 

II.  Literature  Review . 5 

Introduction . 5 

System,  Model  and  Simulation . 5 

Simulation . 7 

Simulation  Classifying . 7 

Discrete  Event  Simulation . 8 

Overview  of  Arena . 1 1 

Description  of  Combat  Identification  (CID) . 12 

Taxonomic  Relationships  Defined . 12 

Outline  of  CID  Process . 13 

Air  to  Ground  (A/G)  CID  Scenario . 14 

Introduction  to  Automatic  Target  Recognition  (ATR) . 14 

Data  Fusion  for  ATR . 16 

Boolean  Fusion  Methodologies  for  ATR  System . 16 

Mathematical  Framework  for  3-Class  Data  Fusion  Experiment . 18 

Concept  of  a  Tactical  UAV  (TUAV) . 20 

Air  Vehicle . 21 

Modular  Mission  Payloads . 22 

Ground  Control  Station . 24 

Ground  Communications . 25 

III.  Methodology . 27 

Introduction . 27 

Model  Description  and  Assumptions . 27 

Simulation  Phases . 31 


vii 


The  ROI . 32 

JSTARs  to  GCS . 34 

TUAV-IN . 36 

TUAV-OUT . 40 

Variables  and  attributes . 40 

Verification  and  Validation . 42 

Conclusion . 42 

IV.  Results  and  Analysis . 43 

General . 43 

Measure  of  Effectiveness . 43 

Simulation  Results . 44 

Comparison  of  different  systems . 48 

Conclusion . 55 

V.  Conclusions  and  Recommendations . 56 

General . 56 

Summary  and  Conclusions . 57 

Recommendations  for  Future  Study . 58 

Appendix  A.  Decision  times  of  System  2  for  computing  the  Joint  Distribution  of 

Decision  time . 59 

Appendix  B.  TPRs  of  System  2  for  computing  the  Joint  Distribution  of  TPR . 60 

Appendix  C.  Decision  times  of  System  3  for  computing  the  Joint  Distribution  of 

Decision  time . 61 

Appendix  D.  TPRs  of  System  3  for  computing  the  Joint  Distribution  of  TPR . 62 

Bibliography . 63 

Vita  . 65 


viii 


List  of  Figures 


Page 

Figure  2.1  Ways  to  study  a  system . 6 

Figure  2.2  Flow  of  control  for  the  next-event  time-advance  approach . 10 

Figure  2.3  Combat  ID  Taxonomy . 13 

Figure  2.4  Combat  ID  Process . 13 

Figure  2.5  Notional  ATR  system  with  Sensors  A&B . 15 

Figure  2.6  Examples  of  various  Boolean  Fusion  Rules  with  Venn  Diagram . 17 

Figure  2.7  Confusion  Matrix  with  Rejection  and  Error  contribution . 18 

Figure  2.8  Air  Vehicle . 21 

Figure  2.9  EO/IR  Payload . 22 

Figure  2.10  Synthetic  Aperture  Radar  Imaging  Concept . 23 

Figure  2.1 1  Ground  Control  Station  (GCS)  -  Interior  View . 25 

Figure  2.12  GCS  Radios  and  Communication  Devices . 26 

Figure  3.1  Cooperative  UAVs  Discriminating  an  object . 30 

Figure  3.2  Entire  Simulation  Model  for  CID  process . 32 

Figure  3.3  Notional  ROI  with  a  target . 33 

Figure  3.4  Object  Allocations . 33 

Figure  3.5  Sub  model  for  collecting  a  Decision  time . 35 

Figure  3.6  Fusion  process  in  our  swarm  system . 39 

Figure  4.1  Accuracy  of  the  systems  as  a  function  of  No.  of  MTIs . 45 

Figure  4.2  Number  of  TUAVs  shot  down  and  Prep  time  as  a  function  of  No.  of 

MTIs . 46 


IX 


Figure  4.3  Decision  and  simulated  mission  timeas  a  function  of  No.  of  MTIs . 47 


X 


List  of  Tables 


Page 

Table  3.1  Typical  Performance  Measures  Associated  with  the  Confusion  Matrix 

Cell,  CM  (row,  col) . 38 

Table  3.2  Variables  and  Attributes  in  the  model . 41 

Table  4.1  Specification  of  a  sensor  used  in  each  system . 44 

Table  4.2  Paired  T-test  for  Accuracy . 50 

Table  4.3  Paired  T-test  for  Number  of  TUAVs  shot  down . 51 

Table  4.4  Paired  T-test  for  TUAV  Prep  time . 52 

Table  4.5  Paired  T-test  for  Decision  time . 53 

Table  4.6  Paired  T-test  for  Simulation  time . 54 

Table  5.1  System  showing  better  performance  in  each  environment . 56 


XI 


COMBAT  IDENTIFICATION 


USING  MULTIPLE  TUAV  SWARM 


I.  Introduction 


1.1.  Background 

During  Operation  Iraqi  Freedom  some  air-to-ground  fratricide  involving  US  pilots 
occurred.  One  of  the  worst  incidents  occurred  when  US  Navy  jets  attacked  Kurdish 
fighters  and  a  group  of  US  Special  Forces  (SOFs)  leading  a  convoy  of  them  in  northern 
Iraq.  US  Special  Forces  sent  two  US  fighter  jets  a  message  by  radio  to  remove  an  Iraqi 
tank  obstructing  their  way.  But  one  of  jets  fired  a  missile  at  the  Special  Forces  and 
Kurdish  fighters,  instead  of  dropping  its  bomb  on  an  Iraqi  tank.  This  incident  resulted  in 
18  Kurds  killed  and  three  SOF  injured.  After  making  an  investigation  into  the  fratricide, 
the  investigator  announced  that  there  was  a  simple  communication  mistake  as  a  cause  of 
incident.  The  radio  of  the  US  navy  jet  that  had  dropped  a  bomb  on  friendly  forces  was 
incompatible  with  the  SOF.  The  SOF  could  communicate  only  with  USAF  aircraft  in  that 
operation.  Furthennore,  although  the  vehicles  carrying  the  friendly  fighters  was  marked 
with  clear  fluorescent-orange,  the  fighter  aircraft  could  not  identify  them,  because  dust 
and  low  clouds  had  obscured  the  pilots’  vision  [1], 

According  to  US  Army  TRADOC  Fratricide  Action  Plan,  “Fratricide  is  the 
employment  of  friendly  weapons  and  munitions  with  the  intent  to  kill  the  enemy  or 
destroy  his  equipment,  or  facilities,  which  result  in  unforeseen  and  unintentional  death  or 
injury  to  friendly  personnel.’’  The  incident  of  fratricide  has  continuously  happened  as  a 
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result  of  warfare  on  a  battlefield.  Especially  in  ground-to-ground  or  air-to-ground 
engagements,  one  of  the  important  tasks  for  the  efficient  conduct  of  modern  warfare  is  to 
preclude  or  reduce  such  misshapes  from  occurring.  Aircrafts  fly  faster  and  Tanks  fire  at 
longer  rages.  Their  precision  and  lethality  have  been  increased  through  advances  in 
technology.  In  addition  the  capability  to  identify  friend-or-foe  has  made  startling 
progress.  Nevertheless,  the  lack  of  target  identification  and  situational  awareness  on  the 
battlefield  occasionally  results  in  fratricide.  As  seen  in  combat  history,  fratricide  will 
never  totally  be  eliminated,  but  the  number  of  occurrences  of  fratricide  can  be  reduced 
via  development  in  technology  for  Combat  Identification  (CID)  along  with  enhanced 
training. 

When  Army  combat  commanders  are  asked  what  has  contributed  most  to  their 
victory  in  Iraq,  they  often  respond  that  it  is  Intelligence,  Surveillance,  and 
Reconnaissance  (ISR).  This  comes  from  the  coordinated  use  of  Unmanned  Aerial 
Vehicles  (UAVs)  (from  Ravens  to  Global  Hawks),  manned  aircraft  carrying  cameras, 
electric  sensors,  and  many  other  assets  for  ISR  [2]. 

In  July  2008,  in  an  interview  with  The  Hill  (a  congressional  newspaper),  Air 
Force  Lt.  Gen.  Michael  Peterson  said,  “What  we  did  is  we  took  an  end-to-end  look  at 
what  we  could  provide  and  what  we  can  deliver  in  terms  of  ISR.  The  highlight  of  that  is 
the  importance  of full-motion  video  to  the  ground  force.  Today  most  of  that  is  done  with 
the  Predator  [ unmanned  aerial  vehicle  (UAV)J.  Global  Hawk  [UA  V]  has  still  images,  but 
we  also  moved  on  with  a  few  aircraft  called  Reaper  [UA  V],  which  is  the  follow-on 
generation  to  the  Predator .”  [3].  UAVs  are  more  often  being  used  for  military  purposes, 
like  search  and  surveillance  in  the  battlefield.  They  play  a  decisive  role  in  information 
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collection  from  hostile  and  unknown  areas. 


1.2.  Research  problem 

In  modern  warfare,  UAVs  are  rapidly  taking  on  a  leading  role  in  traditional  and 
non-traditional  ISR,  to  include  Automatic  Target  Recognition  (ATR).  With  the 
development  of  technology,  the  processors  and  sensors  on  UAVs  have  been  improved, 
however,  advancements  are  still  needed  before  they  can  be  widely  employed  as  the 
primary  or  perhaps  sole  source  for  positive  identification  in  the  CID  process. 

The  CID  process  strongly  depends  on  the  ability  of  the  sensors  to  detennine  the 
identification  of  objects.  Poor  sensor  information  will  result  in  poor  performance  in  the 
CID  process.  Fused  sensors  from  coordinated  UAVs  can  provide  more  information  than 
the  sensor  with  a  single  UAV.  Cost  is  a  driving  factor  for  operating  an  ATR  using 
multiple  UAVs.  The  construction  cost  of  such  an  ATR  system  relies  on  the  quality  of 
sensors,  airframes,  and  many  other  components.  However,  sending  two  or  more 
coordinated  UAVs  with  less  complex  sensors  may  lead  to  an  equivalent  or  even  better 
CID  call  than  sending  a  single  UAV  with  more  sophisticated  sensors  at  a  significantly 
higher  cost.  In  addition,  the  coordinated  UAVs  may  be  capable  of  reducing  the  time  and 
the  number  of  UAVs  needed  to  correctly  discriminate  an  object. 

We  expect  that  a  swanned  system  [4]  incorporating  the  capabilities  of  multiple 
UAVs  to  cooperatively  perfonn  ATR  will  decrease  simulated  mission  time  and  increase 
likelihood  of  making  the  correct  CID  call  for  selected  scenarios. 
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1.3.  Scope 

The  scope  of  this  thesis  is  to  analyze  multi-agent  UAV  swarm  to  understand  the 
effects  of  changing  defined  variables  in  CID  process,  including  True  Positive  Rates 
(TPRs)  in  the  discrimination  process  for  scenarios  with  varying  number  of  enemies.  The 
analysis  includes  construction  of  a  detailed  computer  simulation  of  the  entire  CID  process 
in  Arena  for  a  scenario  with  only  enemies  present.  The  model  is  used  to  explore  major 
factors  that  affect  the  decision  time  and  the  correct  CID  call.  This  analysis  examines  the 
feasibility  of  ATR  using  a  UAV  swarm  compared  to  ATR  using  a  single  UAV  for  a 
variety  of  performance  measures  and  provides  insight  into  developing  a  strategy  to 
improve  ATR  performance  in  such  a  system. 

1.4.  Thesis  Organization 

The  remainder  of  this  document  is  organized  as  follows.  Chapter  2  presents  a 
literature  review  including  an  introduction  of  system,  model,  and  simulation,  an  overview 
of  Arena  as  a  simulation  tool,  a  description  of  CID,  and  an  introduction  of  a  Tactical 
UAV  (TUAV)  and  ATR.  It  also  contains  an  overview  of  a  method  to  perform  fusion  and 
common  technique  used  to  assess  ATR  performance.  Chapter  3  presents  a  methodology 
for  constructing  the  simulation  model  in  Arena.  Chapter  4  provides  the  results  and 
analysis  of  simulation.  Finally,  chapter  5  presents  a  summary  of  contributions  and 
findings  along  with  thoughts  for  the  continuation  of  related  research. 
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II.  Literature  review 


Introduction 

This  literature  review  is  arranged  with  the  following  primary  sections.  Section  2. 1 
provides  a  review  of  the  subjects  related  to  system,  model  and  simulation.  Section  2.2 
provides  an  overview  of  Arena,  a  simulation  software  package,  used  to  build  the  CID 
model.  Section  2.3  provides  a  description  of  CID.  Section  2.4  provides  a  general 
knowledge  of  TUAV  and  ATR.  Finally,  section  2.5  presents  an  overview  of  the  method 
to  perform  fusion  and  common  techniques  used  to  assess  ATR  performance. 

2.1  System,  Model  and  Simulation 

A  system  that  relies  on  the  objective  of  a  particular  study  is  defined  as  a  collection 
of  entities  which  interact  each  other  toward  the  attainment  of  some  logical  end.  The  state 
of  a  system  is  a  collection  of  variables  necessary  to  describe  a  system  at  a  particular  time. 
For  example,  in  a  study  of  a  bank,  if  one  wants  to  decide  the  number  of  tellers  needed  to 
provide  adequate  customer  service,  the  system  can  be  described  as  a  portion  of  the 
customers  waiting  in  line  or  being  served  and  the  tellers.  In  addition,  the  tellers  and  the 
customers  can  be  defined  as  objects  and  examples  of  possible  state  variables  are  the  time 
of  arrival  of  each  customer  and  the  number  of  tellers  in  the  bank  [4:3]. 

A  model  is  defined  as  a  simplified  representation  of  a  system  at  some  particular 
point  in  time  or  space  intended  to  promote  understanding  of  the  real  system.  Modeling, 
especially  scientific  modeling  implies  the  process  of  generating  a  model  as  a  conceptual 
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representation  of  some  phenomenon.  This  conceptual  model  is  used  to  implement 
computer  simulation  that  describes  the  behavior  of  a  system  over  time.  In  general  one 
builds  a  model,  simulates  it,  gets  something  by  simulating  the  model,  revises  the  model 
and  proceeds  the  iteration  till  a  sufficient  level  of  understating  is  obtained. 

Figure  2.1  maps  out  diverse  ways  in  which  a  system  might  be  studied  in  order  for 
one  to  get  some  insight  into  the  interactions  or  relationships  among  various  components, 
or  make  a  prediction  for  the  perfonnance  under  new  conditions  being  set. 


Figure  2.1  Ways  to  study  a  system  [4:4] 


2.1.1  Simulation 
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“Simulation  refers  to  a  broad  combination  of  techniques  and  applications  to 
imitate  the  real  systems.  Simulation  is  usually  implemented  on  computer  by  using  several 
available  software  tools.  Furthermore,  it  can  be  said  that  simulation  is  a  general 
expression  since  the  idea  finds  its  applications  across  many  fields  and  industries”  [5].  If 
the  mathematical  or  logical  relationships  that  constitute  a  model  are  simple  enough, 
mathematical  methods,  e.g.,  probabilistic  theory  or  algebra  would  be  fine  with  one  who 
wants  to  gain  some  understanding  about  behavior  of  the  model.  However,  because  most 
real  systems  are  too  complex,  realistic  models  cannot  be  evaluated  analytically,  and  these 
models  should  be  studied  by  simulation.  Applications  for  simulation  are  huge  and 
various.  Its  examples  are  listed  below  [4:2]. 

•  Designing  and  analyzing  manufacturing  systems 

•  Evaluating  military  weapons  system  or  their  logistics  requirement 

•  Analyzing  supply  chain 

•  Determining  hardware  and  software  requirements  for  a  computer  system 

•  Evaluating  designs  for  service  organizations  such  as  contact  centers,  fast-food 
restaurants,  hospitals,  and  post  offices 

2.1.2  Simulation  Classifying 

Simulation  is  classified  along  three  different  dimensions  as  discussed  below  [4:5]. 

•  Statistic  us\  Dynamic  Simulation  Models.  A  static  simulation  model  represents  a 
system  at  a  specific  time,  or  used  to  represent  a  system  where  time  plays  no  role. 
Monte  Carlo  models  are  good  examples  of  a  static  simulation.  On  the  other  side,  a 
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dynamic  simulation  is  defined  as  a  depiction  of  a  system  where  it  evolves  over 
time,  such  as  a  queuing  system  in  a  bank. 

•  Deterministic  vs.  Stochastic  Simulation  Models.  A  detenninistic  simulation  model 
is  defined  as  a  simulation  model  does  not  include  any  probabilistic  components. 
Once  input  quantities  and  relations  in  detenninistic  models  are  detailed,  the  output 
is  determined.  If  a  simulation  model  contains  some  random  input  components,  it 
is  called  stochastic.  A  stochastic  simulation  model  produces  output  that  is  itself 
random. 

•  Continuous  vs.  Discrete  Simulation  Models.  A  discrete  simulation  model  has  state 
variables  changing  only  at  a  countable  number  of  points  in  time.  These  points  in 
time  are  the  ones  at  which  the  events  occur,  or  change  in  state.  On  the  other  hand, 
if  the  state  variables  change  in  a  continuous  way,  it  is  a  representation  of  a 
continuous  simulation  model. 

2,1.3  Discrete  Event  Simulation 

In  a  discrete  event  simulation,  the  system  operation  is  defined  as  a  chronological 
sequence  of  events.  Each  event  occurs  at  an  instant  in  time  and  marks  a  change  of  state  in 
the  system  [6],  “Though  simulation  has  been  applied  to  a  great  diversity  of  real  world 
systems,  discrete  event  simulation  models  all  share  a  number  of  common  components 
and  there  is  a  logical  organization  for  these  components  that  promotes  the  programming, 
debugging,  and  future  changing  of  a  simulation  model’s  computer  program”  [4:9].  In 
most  discrete-event  simulation  models,  the  following  components  are  found. 

•  System  state :  The  collection  of  state  variables  necessary  to  describe  the  system  at 


a  particular  time. 

•  Simulation  clock :  A  variable  giving  the  current  value  of  simulated  time. 

•  Event  list:  A  list  containing  the  next  time  when  each  type  of  simulated  Time. 

•  Statistical  counters:  Variables  used  for  storing  statistical  infonnation  about 
system  perfonnance. 

•  Initialization  routine:  A  subprogram  to  initialize  the  simulation  model  at  time 
zero. 

•  Time  routine:  A  subprogram  that  detennines  the  next  event  from  the  vent  list  and 
then  advances  the  simulation  clock  to  the  time  when  that  event  is  to  occur. 

•  Event  routine:  A  subprogram  that  updates  the  system  state  when  a  particular  type 
of  event  occurs  (there  is  one  event  routine  for  each  event  type). 

•  Library  routines:  A  set  of  subprograms  used  to  generate  random  observations 
from  probability  distributions  that  were  determined  as  part  of  the  simulation 
model. 

•  Report  generator:  A  subprogram  that  computes  estimates  of  the  desired  measures 
of  performance  and  produces  a  report  when  the  simulation  ends. 

•  Main  program:  A  subprogram  that  invokes  the  timing  routine  to  determine  the 
next  event  and  then  transfers  control  to  the  corresponding  event  routine  to  update 
the  system  state  appropriately. 

Figure  2.2  shows  the  logical  relationships  among  these  components.  First  of  all, 
the  initialization  routine  is  invoked  in  the  main  program,  and  the  simulation  starts  at  the 
time  zero,  where  the  simulation  clock  is  set  to  zero,  the  simulation  initializes  the  system 
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state,  statistical  counter,  and  event  list.  Then,  the  main  program  takes  control  in  order  for 
it  to  invoke  the  timing  routine  to  decide  the  next  event  time.  If  event  type  i  is  the  next  to 
occur,  the  time  routine  advances  the  simulation  clock  to  the  time  that  an  event  i  will 
occur,  and  event  routine  i  is  invoked  by  the  main  program.  After  the  system  state  and  the 
statistical  counters  gathering  information  about  performance  of  system  are  updated,  the 
occurrence  time  of  future  events  are  generated,  and  the  information  is  added  to  the  event 
list.  Then,  if  the  stopping  conditions  are  satisfied,  the  estimates  (from  the  statistical 
counters)  of  performance  measure  and  a  report  are,  respectively,  computed  and  made  by 
the  report  generator.  On  the  other  hand,  if  the  conditions  for  termination  are  not  enough, 
the  main  program  is  invoked  from  the  event  routine  i  and  all  processing  is  repeated  until 
the  stopping  conditions  are  met. 
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Figure  2.2  Flow  of  control  for  the  next-event  time-advance  approach  [4:10] 
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2.2  Overview  of  Arena 


Arena  10  is  used  for  the  development  and  analysis  of  this  research.  This 
simulation  and  automation  software  using  the  SIMAN  processor  and  simulation  language 
is  developed  by  Rockwell  Automation.  Arena  is  widely  used  to  simulate  a  company’s 
process,  such  as  manufacturing  plants,  to  analyze  its  current  performance  as  well  as 
possible  changes  that  could  be  made.  By  accurately  simulating  a  process,  a  company  can 
see  the  outcome  of  changes  without  implementing  them  in  real-time,  thus  saving  valuable 
time  and  resources. 

With  Arena,  the  user  can  build  their  model  by  dragging  module  representing 
process  or  logic  from  the  Project  Bar  into  the  Model  Window  Flowchart  and  placing  it 
and  simulate  the  performance  of  the  system  to  understand  complicated  relationships  and 
figure  out  opportunities  for  improvement.  Besides,  since  it  can  visualize  the  operation 
with  dynamic  animation  graphic,  this  feature  gives  the  user  more  clear  understanding  and 
helps  the  user  check  model  verification.  Finally,  to  decide  the  best  way,  the  user  analyzes 
how  the  system  will  perform  in  its  various  configurations  and  under  a  myriad  of  possible 
alternatives  [7]. 

Arena  itself  has  three  additional  tools  for  analyzing  data  or  performance 
measures.  Firstly,  the  input  analyzer  is  used  to  determine  the  best  fit  of  probability 
distribution  functions  to  input  data.  Besides,  it  can  generate  random  data,  such  as  a  set  of 
inter-arrival  time  or  process  time,  to  be  analyzed.  The  output  analyzer  component  of 
Arena  is  used  to  display  output  data  so  that  it  allows  the  user  to  view  and  analyzed  his 
data  quickly  and  easily.  Lastly,  if  the  simulation  model  is  completed,  validated  and 
configured  properly  for  use  by  the  process  analyzer,  the  process  analyzer  is  used  to 
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compare  the  outputs  from  validated  models  based  on  different  model  inputs,  such  as 
different  process  times  and  possible  number  of  teller  in  a  bank  and  etc.,  or  scenarios. 

2.3  Description  of  Combat  Identification 

Sensors,  communications  and  computation  systems  to  provide  situational 
awareness  of  battle  space  are  combined  in  network  centric  warfare  (NCW).  One  of 
elements  of  the  awareness  is  known  as  CID  to  indentify  friendly  combat  entities  or 
enemies. 

CID  is  defined  as  a  process  of  attaining  accurate  characteristics  of  objects 
detected  in  the  battle-space  and  identifying  the  objects  as  friends,  neutrals  and  enemies. 
CID  must  be  performed  in  the  all  of  mission  area  including  air,  land  and  sea  in  order  for 
commanders  to  timely  decide  their  COA  on  enemy  action  and  to  effectively  employ  their 
forces  and  weapons  and  to  minimize  friendly  force’s  casualties. 

2.3.1  Taxonomic  Relationships  Defined 

As  shown  in  Figure  2.3,  identity  can  be  described  by  seven  states.  Each  object 
detected  is  identified  as  one  of  them  consisting  of  hostile,  neutral  and  friendly.  It  refers  to 
where  the  object  is  detected  and  which  type  of  object  is  at  different  levels  of  detail  from 
platform  type  like  fighter.  Nationality  implies  that  which  country  the  aircraft  belongs  to. 
Finally,  activity  mission  tells  for  what  the  aircraft  is  doing  such  as  air  patrol  or  strike. 
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Figure  2.3  Combat  ID  Taxonomy  [9] 


2.3.2  Outline  of  CID  Process 

The  entire  concept  of  CID  is  shown  in  Figure  2.4.  The  “Direct”  ID  source  comes 
from  sensors  equipped  with  the  platform  of  shooter  but  the  “Indirect”  ID  source  refers  to 
off-board.  Single  or  multi  sources  of  ID  infonnation  each  have  some  probability  of 
correct  ID  are  fused  to  provide  an  overall  statistical  declaration  of  the  identity.  Then,  the 
engagement  rules  are  applied  and  if  it  is  within  threshold  for  statistical  declaration,  firing 
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Figure  2.4  Combat  Identification  Process  [10] 
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2.3.3  Air  to  Ground  (A/G)  CID  Scenario 

Though  there  are  several  CID  scenarios  such  as  Air  to  Air  and  Air  to  Ground  and 
etc,  this  section  will  cover  only  A/G  scenario  of  CID,  because  this  research  is  a  flavor  of 
A/G  CID.  The  A/G  CID  architecture  consists  of  three  platforms:  a  Forward  Air 
Controller  (FAC)  being  assumed  to  a  ground  FAC,  A/G  shooter  being  a  fixed-wing 
aircraft,  and  A/G  surveillance  by  Off-board  sources  or  UAVs.  Each  of  the  platforms  is 
connected  to  each  of  the  other  platforms  to  pass  data  in  all  nodes.  Given  an  object 
detected,  a  fusion  algorithm  is  applied  to  combine  various  sources  from  cooperative  and 
non-cooperative  systems.  If  it  is  declared  as  a  target,  CID  information  will  be  transferred 
to  a  warfighter  to  let  him  know  all  information  of  the  target.  After  all,  he  will  shoot  the 
target  and  completes  his  mission. 

2.3.4  Introduction  to  Automatic  Target  Recognition 

As  mentioned  above,  Combat  ID  uses  both  cooperative  system  and  non- 
cooperative  identification  methods.  Identification  Friend  or  Foe  (IFF)  system  is  an 
example  of  cooperative  systems.  This  system  inquires  it  using  electronic  communication 
between  two  friendly  systems  to  identity  a  potential  target.  If  there  is  no  feedback,  non- 
cooperative  means  must  be  made. 

Non-cooperative  means  can  be  performed  by  a  man-in-the-loop,  autonomously  by  an 
identification  system,  or  by  both.  One  possible  man-in-loop  mean  is  for  the  pilot  to 
visually  confirm  the  potential  target  before  engagement.  An  Automatic  Target 
Recognizer  (ATR)  is  considered  to  be  a  case  of  non-cooperative  Combat  ID  performed 
by  an  autonomous  system  [11 :2] . 
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Military  ATR  systems  have  been  around  since  the  early  1960’s  [12].  Laine  [11] 
states,  “improved  ATR  systems  would  help  streamline  the  Combat  ID  process  and  allow 
the  USAF  to  use  Global  Hawk  at  more  than  the  one -third  capacity  used  during  Operation 
Iraqi  Freedom  (OIF).” 

Figure  2.5  shows  a  notional  ATR  system  with  sensors.  One  or  multi  sensors  on 
the  same  of  different  platforms  are  assigned  to  the  Region  of  Interest  where  potential 
targets  can  be  located.  Then,  the  system  performs  detecting,  tracking  and  classifying 
them.  The  ATR  system  is  suggested  that  it  contain  a  minimum  of  three  output  labels: 
Target,  Non-Target,  and  Non-declaration.  According  to  USAF  doctrine,  a  desired  level  of 
confidence  in  declaring  the  object  is  needed  before  making  a  lire  decision  [13],  [14]. 
When  “non-declaration”  is  labeled  due  to  lack  of  confidence,  the  ATR  system  may  keep 
watching  to  get  more  information.  In  these  situations,  the  current  ATR  system  takes 
multiple  looks  of  the  same  target  within  limited  time,  since  other  platforms  may  not  be 
available  to  help  in  the  ID  process  [1 1:5]. 
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Figure  2.5  Notional  ATR  System  with  Sensors  A  &  B  Collecting  Data  through  Time  [11] 
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2.4  Data  Fusion  for  Automatic  Target  Recognition. 

Lots  of  quantitative  techniques  are  available  to  implement  fusion  at  various  levels. 
It  is  important  for  one  to  find  the  best  technique  for  a  given  application.  All  the  data 
fusion  communities  do  not  agree  about  which  fusion  methodology  is  best  for  the 
application  [15].  However,  if  the  individual  sensor  data  is  refined  to  create  a  class  label, 
then  the  Boolean  voting  logic  is  considered  as  a  standard  fusion  methodology  to  figure 
out  a  single  class  estimate  [16],  [17],  [18]. 

2.4.1  Boolean  Fusion  Methodologies  for  ATR  System 

Boolean  rules  can  be  used  as  a  method  of  combining  output  labels  of  distinctive 
identification  systems.  The  followings  are  examples  of  Boolean  fusion  rules. 

•  Logical  AND.  If  and  only  if  all  of  the  sensor  labels  denote  the  target  is  a  "Hostile", 
the  unknown  objective  is  identified  as  a  Hostile  target.  This  rule  may  be  called  as 
the  AND  rule. 

•  Logical  OR.  If  any  of  the  sensors  classify  it  as  a  "Hostile",  then  the  system 
concludes  an  object  is  a  "Hostile."  This  rule  may  simply  be  referred  to  as  the  OR 
rule. 

•  Majority  Vote.  A  majority  of  the  sensors  is  required  to  determine  the  target  as  a 
"Hostile."  For  example,  2  or  more  "Hostile"  label  are  needed  for  a  three  sensor 
suite  to  declare  "Hostile." 

•  Sensor  Corroboration.  The  fusion  logic  related  to  sensor  corroboration  requires 
one  of  the  sensor  not  only  to  declare  a  target  as  a  "Hostile"  but  also  to  corroborate 
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this  label  with  one  of  others.  As  mentioned  by  Hill  (2003),  such  a  fusion  rule  may 
be  appropriate  when  sensors  do  perform  different  function  in  discriminating  an 
object. 

•  Sensor  dominance.  This  fusion  logic  may  be  appropriate  in  the  case  that  one 
sensor  performs  better  than  others.  For  example,  sensor  A  has  high  confidence 
and  declares  a  fused  "Hostile"  label  regardless  of  the  labels  from  sensor  B  and  C. 

Figure  2.6  for  the  use  of  three  sensors  (SA,  SB  and  Sc)  depicts  an  illustration  of 
Boolean  logic  fusion  rules  for  the  use  of  three  sensors.  Positive  declarations  of  “Hostile” 
targets  are  represented  by  the  grey  areas  for  each  of  the  Boolean  fusion  rules. 
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Figure  2.6  Examples  of  Various  Boolean  Fusion  Rules  with  Venn  Diagrams  [1 1:45] 
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2.4.2  Mathematical  Framework  for  3-Class  Data  Fusion  Experiment 

Various  classification  problems  have  been  modeled  at  the  top-level  using  2- 
classes.  An  ATR  system  declaring  an  unknown  object  as  a  target  or  non  target  is  an 
example  showing  a  2-classes  model  above.  However,  before  an  ATR  system  declares  an 
unknown  object  as  a  target  and  it  is  engaged,  a  minimum  level  of  confidence  is  required 
by  the  USAF.  Consequently,  three  output  classes  including  “Target,”  “Non-target”  and 
“Non-declaration”  are  required  to  explain  for  those  cases  when  the  confidence  is  not 
satisfied.  Figure  2.7  shows  a  sample  of  2-class  confusion  matrix  with  a  “Non¬ 
declaration,”  with  a  column  for  each  labeled  class  and  a  row  for  each  true  class.  For  most 
applications,  warfighters  perfonn  horizontal  analysis,  but  engineers  are  concerned  with 
vertical  analysis  of  the  confusion  matrix  yields  the  estimates  of  error  from  the  number  of 
class  labeled.  The  error  rates  are  calculated  as  conditional  probabilities  using  Bayes  rule 
with  other  prior  probabilities  of  class  membership  (denoted  TV  and  P/  ). 
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The  critical  and  non  critical  errors  will  be  defined  as  below. 


•  Probability  of  a  Critical  Error:  the  probability  of  classifying  a  Friend  as  "Target", 
such  a  case  like  fratricide, 

Number  of  Friends  declared  as  "Target" 


P(ECR )  = 


Total  nuber  of  "Target"  declarations 


PfPfp 


(2.1) 

PpPpp  +  PpPpp 

•  Probability  of  a  Non  Critical  Error:  the  probability  of  classifying  an  Enemy  as  a 
Friend,  for  example  lost  opportunities  to  engage  the  enemy, 

Number  of  Targets  declared  as  "Friend" 


P(Enc )  = 


Total  nuber  of  "Friend"  declarations 


PtPfn 


(2.2) 


PfPtn  +  PtPfn 

and  the  probabilities  of  False  Negatives  and  True  Negatives  are,  PFN  —  PFN(6)  —  1  — 
PTp(9 )  and  PTN  —  PTN(6j  =  1  —  PpP(0).  If  it  is  assumed  that  all  objects  belong  to  one 
of  the  true  classes,  PDec,  the  probability  of  declaration,  can  be  used  as  a  performance 
measure  of  the  “Non-declaration”  labels.  The  probability  of  rejecting  a  sample  is  related 
as :PREJ  =  1  —  Ppec-  The  probability  of  a  declaration  is  then: 


•  Probability  of  a  Declaration:  the  probability  of  either  class  being  declared  "ND" 

Number  of  objects  declared  as  "ND" 

Dec  total  objects  evaluated 

—  PpPyp  +  PfPuf  >  (2-3) 

where  PUT  =  P("ND"\T)  and  PUF  =  Pf'ND"\F).  Given  all  probabilities  calculated  from 

test,  P  =  P  will  be  assumed  for  the  remainder. 
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2.5  Concept  of  Tactical  Unmanned  Aerial  Vehicle  (TUAV) 

TUAV  is  designed  to  be  a  ground  maneuver  brigade  commander’s  UAV, 
providing  him  with  a  number  of  benefits  to  include:  enhanced  enemy  situational 
awareness,  a  target  acquisition  capability,  battle  damage  assessment  (BDA),  and 
enhanced  battle  management  capabilities  (friendly  situation  and  battlefield  visualization). 
TUAV  gives  the  commander  “dominant  eye”  allowing  him  to  see  into  area  where  the 
commander  does  not  want  to  send  ground  reconnaissance  elements  or  manned  aerial 
platforms.  It  can  be  linked  to  and  cued  by  sound  IPB  and  wide  area  sensors  such  as 
JSTARS  Common  Ground  Station  (CGS),  Artillery  Counter  Mortar/Battery  Radars,  and 
Forward  Area  Air  Defense  System  (FAADS)  for  distribution  via  intelligence  channels. 

The  system  is  designed  for  ease  in  launching,  operating,  recovering,  and 
maintaining  with  minimal  training,  logistics,  and  personnel.  To  reduce  its  footprint,  tear 
down  in  rapid,  deploy  and  set  up,  and  minimize  impact  on  brigade  combat  service 
support  (CSS)  resources,  it  presents  a  small  profile  in  battlefield. 

A  TUAV  system  consists  of  four  basic  components:  Ground  Control  Stations 
(GCS)  and  related  equipment,  Air  Vehicles  (AV),  Modular  Mission  Payloads  (MMP), 
and  communications.  The  TUAV  baseline  is  to  provide  12  hours  of  continuous 
operations  within  a  24-hour  period.  For  no  more  than  three  consecutive  days,  the  system 
is  capable  of  surge  operations  for  18  hours  per  24-hour  period,  and  the  following  day  is 
limited  to  8  hours  of  operation.  A  full  baseline  requires  a  crew  of  approximately  22  for 
operation  and  maintenance  at  the  operational  tempo  above. 
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2.5.1  Air  Vehicle 


The  air  vehicle  (Figure  2.8),  constructed  of  composite  materials  and  powered  by  a 
rotary  engine,  is  a  mid-wing  monoplane  with  a  twin  boom  empennage  supporting  an 
inverted-V  tail.  The  TUAV  system  has  four-hour  endurance  with  a  range  of  50  kilometers 
from  the  launch  and  recovery  site.  It  is  operated  with  clear  line  of  sight  between  the  air 
vehicle  and  ground  data  terminal/portable  ground  data  terminal  [19].  Though  it  is  not 
designed  to  meet  the  requirements  of  low  signature,  due  to  composite  materials  and  its 
small  size,  it  can  reduces  signature  characteristics  and  is  not  visually  detectable  from 
range  exceeding  4,000  ft  and  not  audible  from  ranges  exceeding  2,000  ft.  It  is  capable  of 
operating  during  less  than  ideal  weather  conditions,  such  as  moderate  rain  condition, 
within  its  radius  of  action.  Nominal  operating/survivable  altitudes  for  day  and  night 
operations  are  respectively  from  8,000  to  10,000  feet  Above  Ground  Level  (AGL)/from 
6,000  to  8,000  feet  AGL.  Heavy  icing,  precipitation,  or  high  surface  winds  may  prevent 
launches  or  operations  in  some  areas. 


Figure  2.8  Air  Vehicle  [20] 
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2.5.2  Modular  Mission  Payloads  (MMPs) 

The  baseline  sensor  of  the  TUAV  is  the  Electro-Optic/Infrared  (EO/IR)  payload 
(Figure  2.9).  The  payload  is  a  multi-mode,  Forward  Looking  Infrared  (FLIR)  /  Line 
Scanner  /  TV  sensor.  It  is  required  to  have  resolution  enough  to  detect  and  recognize  an 
Armored  Personnel  Carrier  (APC)  sized  target  at  the  operational  altitudes  of  day  and 
night  operation  and  at  survivable  standoff  ranges  (3-5km)  from  imaged  target.  In  EO  and 
IR  mode,  the  requirements  are  respectively  for  80  percent  probability  of  detection  at  3.8 
km  and  70  percent  of  that  at  the  targeted  range  of  3.5  km  [18].  It  is  capable  of 
autonomous  preplanned  operation  and  instantaneous  retasking  throughout  a  mission.  In 
addition,  it  can  provide  continuous  zoom  capabilities  when  in  EO  mode  and  multiple 
Fields  of  View  (FOV)  when  in  IR  and  slew  360  degrees. 


Figure  2.9  EO/IR  Payload  [20] 


The  payload  of  secondary  priority  is  a  Synthetic  Aperture  Radar/Moving  Target 
Indicator  (SAR/MTI).  This  payload  increases  situational  awareness  by  providing  high- 
resolution  imagery  enough  to  detect  and  recognize  APC  sized  target  at  operational 
altitudes  and  survivable  standoff  range  in  this  situation,  e.g.,  adverse  weather  or 
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battlefield  obscurants.  The  MTI  adjunctive  to  Joint  Surveillance  and  Target  Attack  Radar 
System  (JSTARS)  serves  as  an  immediate  cure  to  potential  threat  activity.  If  it  detects 
and  recognizes  threat  activity,  the  activity  can  be  confirmed  by  onboard  EO/IR  or  SAR 
sensors. 


Figure  2.10  Synthetic  Aperture  Radar  Imaging  Concept 

Michael  P.  Farmer  [21:69]  states,  “A  TUAV  payload  that  incorporates  a  SAR, 
which  is  one  of  the  future  payloads  being  designed  for  the  TUAV,  combined  with  an 
ATR  system  at  the  GCS,  could  provide  this  solution  (to  reduce  /  remove  the  role  of  the 
man  in  the  loop  in  the  CID  process)  in  the  near  future.”  According  to  Sandia  National 
Faboratories  [23],  the  expected  appearance  of  target  vehicles  in  SAR  imagery  are 
modeled  and  quantified.  Match  metrics,  derived  from  mathematical  principles,  measure  a 
level  of  agreement  between  target  models  and  potential  targets  detected  in  new  SAR 
imagery. 

The  TUAV  Communications/Data  Relay  payload  is  used  to  provide  Very  High 
Frequency/Ultra  High  Frequency  (VHF/UHF)  beyond  line  of  sight  relay  for 
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communications,  supporting  extended  range  operations  of  Army  XXI  with 
communication  while  operating  on  board  the  TUAV. 

2.5.3  Ground  Control  Station  (GCS) 

A  GCS  is  designed  for  two  primary  functions.  The  first  primary  function  is  to 
control,  track,  and  operate  the  AV.  Second,  it  is  used  to  manipulate  the  payload,  receive, 
and  process  telemetry  and  video  downlinks.  In  addition,  it  can  call  for  and  adjust  indirect 
fire. 

TUAV  system  has  two  GCSs,  each  in  a  High  Mobility  Multipurpose  Wheeled 
Vehicle  (HMMWV)  mounted  Command  and  Control  (C2)  shelter.  The  GCS  consists  of 
two  operator  positions,  an  air  vehicle  operator  position  and  a  Mission  Payload  Operator 
(MPO)  position.  Since  both  positions  are  identical  in  capability,  functions  can  be 
transferred  to  either  when  one  operator  position  fails.  It  can  not  only  control  and 
communicate  with  one  AV  at  a  time  but  also  control  another  AV  while  another  GCS 
prepares  its  AV  for  launching  or  recovering.  As  long  as  there  is  line  of  sight  path  between 
the  Ground  Data  Terminal  (GDT)  and  the  AV,  the  AV  is  controlled  through  the  GDT  to 
distances  up  to  50  kilometers. 
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Figure  2. 1 1  Ground  Control  Station  (GCS)  -  Interior  View  [20] 

2.5.4  Ground  Communications 

The  GCS  provides  easy  interface  to  the  existing  secure  command,  control, 
communications,  computers,  and  intelligence  (C4I)  architecture.  It  includes  Common 
Ground  Station  (CGS),  Advanced  Field  Artillery  Tactical  Data  System  (AFATDS),  All 
Source  Analyses  System  (ASAS),  Forward  Area  Air  Defense  System  (FAADS),  and 
Army  Airspace  Command  and  Control  (AC).  Various  communication  systems 
including  secure  voice,  electronic  dissemination,  and  video  in  the  GCS  are  used  for 
intelligence  reports.  Secure  communications  and  intelligence  dissemination  are  provided 
through  the  standard  DoD  tactical  (VFIF  and  UHF)  radios,  Mobile  Subscriber  Equipment 
(MSE),  and  the  Tactical  Local  Area  Network  (TACLAN).  TUAV  communications  must 
interface  with  selected  standard  DoD  C4I  systems,  National  Security  Agency  approved 
encryption  systems,  and  etc. 
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The  tactical  communication  system  gives  TUAV  tactical  users  integrated 
communication  for  mission  support  and  communication  between  GCSs.  Radio 
communications  between  operators  in  GCS,  external  system  users,  and  support  units  are 
performed  through  Single  Channel  Ground  and  Airborne  Radio  System  (SINCGARS) 
radios.  The  Single  Unit  Transceiver  radio  (SUT)  provides  external  voice  communications 
on  the  flight  line. 


SINCGARS  SIP 

HMMWV  Cab  VHF 


TO  WHOM  Other  Unit  Personnel  in  C  onvoy  or  Movement 
PURPOSE  Internal  02  Movement  or  Convoy  Coordination 


TO  WHOM:  MI  BN  Ops  Met.  ACT.  CGS.  3DE  O&I 

PL'RPO  SE  C2.  Mission  T  a  sitings.  Tactical  Message  Repotting 


SINCGARS  SIP 
(Handheld) 


TO  W  HOM  LR  A  Maintenance  Personnel 
PURPOSE  Internal  C— .  Mis  ston  C  coicilliatic  11 


TO  WHOM:  Other  GC  S  and  TCS 
PURPOSE:  Internal  C2,  Mission  Coordination 


TO  WHOM:  AiC  (Military  &  C  ommercial).  Other  Aircraft 
PURPOSE:  Airspace  Coordination 


TO  WHOM:  MSE  Node.  Commercial  Phone  Lines 
PU  RPOSE:  C  2.  TACCOM  Mission  Taskmgs 


TO  WHOM:  Other  EPLRS  Stations 
PU  RPOSE:  GPS  Data,  AV  Location.  C2 


TO  WHOM  BDE  TOC  (ACT,  CGS,  AFATDS,  A2C2).  and 
MSE  via  TACLAK 

PURPOSE  C  2.  Mission  Tasking.  Tacncal  Message  Repotting 


TO  WHOM:  BDE  TOC  (ACT,  CGS,  AFATDS,  A2C2),  MSE 
PURPOSE .  Pavload  Video 


TO  WHOM  BDE  TOC  (ACT.  CGS,  AFATDS.  A2C2),  MSE 
PU  RPOSE  AV  &  Target  Telemetry 


Figure  2.12  GCS  Radios  and  Communications  Devices  [19] 


Telephones  can  be  also  used  for  communication  between  the  TUAV  Control 
Shelters,  Mobile  Maintenance  Facility,  and  system  users.  A  tactical  telephone  as  a  part  of 
the  TUAV  system  is  capable  of  handing  digital  data  and  voice  communication.  There  are 
two  telephone  networks  available:  Mobile  Subscriber  Equipment  (MSE)  for  telephone 
(voice/data)  communication  and  one  fiber  optic  net  (Ethernet)  for  intra  shelter  voice/data 
communication. 
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ITT.  Methodology 


Introduction 

Chapter  3  presents  our  methodology  for  analyzing  TUAV  swarm  with  low-value 
sensors  versus  a  single  TUAV  with  a  high-value  sensor  and  how  our  simulation  model 
was  constructed.  This  chapter  consists  of  four  sections;  the  first  provides  a  conceptual 
model  description  and  assumptions  for  the  simulation  model,  the  second  section  focuses 
on  the  steps  of  building  the  simulation  model  using  Arena,  the  third  covers  specific 
variables  and  statistics  used  for  our  study,  and  finally  the  fourth  section  discusses 
verification  and  validation  of  the  model. 

3,1  Model  Description  and  Assumptions 

Our  CID  process  begins  with  a  wide  area  sensor’s  (such  as  JSTARS)  reception  of 
Moving  Target  Indicator  (MTI)  data.  This  data  from  the  JSTARS  platfonn  is  transferred 
to  the  JSTARS  CGS  located  at  a  Brigade  Tactical  Operations  Center  (TOC).  A  decision 
maker,  such  as  Brigade  Commander,  Executive  Officer,  or  Operational  Officer,  at  the 
TOC  has  to  decide  if  he  develops  the  MTI  item  further  or  not.  If  he  decides  to  further 
develop  it,  he  may  use  one  of  their  internal  assets  (i.e.,  TUAVs  or  ground  reconnaissance 
units)  to  gain  more  information  on  the  MTI  or  request  additional  intelligence  an  outside 
agency  such  as  satellite  imagery.  If  tasked  to  the  TUAV,  the  item  is  routed  to  a  GCS.  All 
TUAVs  (the  single  system  or  the  two  system  swann)  are  assumed  to  use  the  same 
platform  and  assumptions  for  preparation  time,  and  time  to  travel  to  target  area,  etc.  The 
only  modeled  difference  is  in  terms  of  better  (at  a  significantly  higher  cost)  sensor 
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capabilities  on  the  single  system  TUAV.  In  the  case  of  single  TUAV  system,  on  receiving 
an  MTI,  a  GCS  prepares  to  launch  its  TUAV  and  send  it  to  a  mission  area.  After 
preparation,  a  TUAV  moves  to  the  mission  area  and  begins  the  discrimination  process  in 
a  Region  of  Interest  (ROI).  We  define  the  discrimination  process  as  the  TUAV  declaring 
(labeling)  the  MTI  data  as  an  enemy  (target)  or  not.  For  this  study  every  MTI  is 
considered  to  be  a  true  enemy  and  all  MTIs  occur  within  the  defined  ROI.  Once  in  a 
position  to  begin  its  mission  execution,  a  TUAV  may  be  shot  down  by  air  defense 
weapons  of  an  enemy.  This  is  modeled  through  use  of  a  defined  effective  area  for  an 
enemy  air  defense  system  where  a  TUAV  will  have  a  probability  of  being  shot  down 
within  this  area.  When  a  TUAV  is  shot  down,  another  TUAV  will  be  prepared  and 
launched  to  the  current  MTI  or  to  respond  to  additional  MTI  data.  On  the  other  hand,  a 
functioning  TUAV  proceeds  to  location  of  current  MTI  and  declares  whether  a  target  is 
present  or  not  after  reviewing  the  on-board  sensor  information.  Then,  if  there  are  no 
additional  MTIs  that  need  to  be  developed,  the  TUAV  returns  to  its  Launch  &  Recovery 
(L/R)  site  and  waits  for  new  MTIs. 

With  our  swann  system,  two  TUAVs  are  launched  together  to  perform  this 
mission.  Once  the  MTI  item  is  transferred  from  a  TOC  to  a  GCS,  two  TUAVs  are 
prepared  for  a  mission  and  move  to  the  mission  area.  For  our  study  the  CID  process  is 
modeled  in  the  following  way:  Given  that  each  MTI  is  a  target,  once  TUAV  1  (or  TUAV 
2)  declares  a  target  is  present,  this  declaration  result  of  TUAV  1  (or  TUAV  2)  is 
transferred  to  the  neighboring  TUAV  2  (or  TUAV  1).  On  receiving  the  information,  the 
TUAV  2  (or  TUAV  1)  stops  its  discrimination  process.  Finally,  the  object  is  declared 
(labeled)  as  an  enemy  by  our  fusion  algorithm.  If  TUAV  1  finishes  reviewing  the 
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information  on  the  ROI  from  its  onboard  sensor  earlier  than  TUAV  2  and  does  not 
declare  a  target  is  present,  the  declaration  result  of  TUAV  1  is  transferred  to  TUAV  2  and 
TUAV  2  continues  the  discrimination  process.  If  TUAV  2  declares  a  target  is  present,  our 
fusion  rule  returns  a  target  is  present  in  the  ROI.  If  both  TUAVs  fail  to  declare  the  MTI 
as  a  target,  our  fusion  algorithm  returns  no  target  present  in  the  ROI.  In  either  case,  both 
TUAVs  either  return  to  their  L/R  site  or  respond  to  additional  MTI  data. 

Either  or  both  TUAVs  can  be  shot  down  by  the  antiaircraft  weapon  system  of  an 
enemy  while  executing  a  mission.  If  one  TUAV  is  shot  down  in  a  swarm,  the  remaining 
TUAV  continues  its  mission  alone  until  a  fresh  TUAV  is  ready.  If  both  TUAVs  are  shot 
down,  two  fresh  TUAVs  are  prepared  one  by  one.  As  soon  as  the  first  TUAV  is  ready  it 
is  launched  and  perfonns  its  mission  alone  until  the  second  TUAV  is  ready  and  launched. 

Multiple  TUAVs  fly  in  formation  in  response  to  each  MTI  item.  In  a  swarm,  one 
is  a  leader  TUAV  and  another  is  a  follower  TUAV.  Therefore  the  flight  speed  and 
position  of  a  follower  TUAV  depend  only  on  those  of  a  leader  TUAV.  Figure  3.1 
illustrates  multiple  TUAVs  coordinating  to  discriminate  an  object. 
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Figure  3.1  Cooperative  UAVs  Discriminating  an  Object 


Collision  avoidance  between  TUAVs  is  not  an  issue  in  this  research.  An  efficient 
CID  strategy  is  to  obtain  a  correct  CID  call  (in  our  case  enemy)  in  a  minimum  time  by 
cooperating  and  coordinating  with  other  TUAVs.  Because  cooperation  can  be  achieved 
by  communication  with  other  TUAVs,  we  assume  that  TUAVs  in  a  swarm  communicate 
with  each  other  for  cooperation  while  performing  discriminating  tasks. 

We  assume  that  each  TUAV  is  provided  with  the  following  equipment  to  enable  it 
to  perform  ATR:  wireless  communication  capability  for  sending  and  receiving 
information  from  other  TUAVs  and  the  ground  station,  one  image  sensor  capable  of 
capturing  snapshots  of  the  area  over  which  the  TUAV  flies,  a  global  positioning  system 
(GPS)  that  returns  the  3-d  coordinates  of  the  TUAV’s  current  position,  image  processing 
software  that  enables  the  TUAV  to  discriminate  objects  obtained  by  its  image  sensors  as 
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a  potential  target.  In  a  swarm  system,  each  TUAV  performs  its  mission  in  formation. 
However,  the  declaration  of  a  TUAV  depends  only  on  its  individual  onboard  sensor 
information. 

In  reality,  a  decision  maker  at  a  Brigade  TOC  has  many  options  to  further  develop 
a  MTI  item.  However,  we  consider  only  one  option,  TUAV  tasking,  is  available  to  him  to 
simplify  this  CID  model  and  focus  on  the  performance  of  a  swarm  system.  In  addition, 
though  four  TUAVs  are  typically  available  at  the  level  of  Brigade,  there  is  no  limitation 
on  number  of  TUAVs  available  in  our  model.  Our  model  also  does  not  consider  times 
required  to  upload  and  download  information  on  a  MTI  or  communication  problems 
between  TUAVs. 

To  perform  fusion,  Boolean  logic  is  used  in  this  research.  Though  it  is  easy  to 
implement,  Robinson  and  Aboutalib  [24]  show  that  Boolean  fusion  for  decision  labels  is 
suboptimal  for  two  or  more  sensors  when  each  sensor  is  optimized  independently. 
However,  the  assumption  of  independence  provides  us  with  a  reasonable  estimate  of 
system  performance  (that  is  also  easily  implemented  in  our  model)  using  Boolean  fusion 
for  decision  labels. 

3.2  Simulation  Phases 

With  our  given  conceptual  model  of  the  CID  process,  there  are  many  different 
ways  to  incorporate  the  objects  of  interest  in  a  discrete  event  simulation  model.  In  our 
CID  model,  a  Moving  Target  Indicator  (MTI)  is  defined  as  an  entity.  A  MTI  represents  a 
radar  presentation  which  shows  only  targets  which  are  in  motion.  Signals  from  stationary 
targets  are  subtracted  out  of  the  return  signal  by  the  output  of  a  suitable  memory  circuit. 
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MTIs  are  created  one  at  a  time  with  a  selected  random  amount  of  time  between  each  and 


are  later  randomly  placed  within  the  defined  ROI.  A  TUAV  is  modeled  as  a  resource  with 
an  unlimited  number  available.  However,  only  one  TUAV  can  be  in  active  in  the  system 
for  the  single  TUAV  system,  while  up  to  two  TUAVs  can  be  active  in  the  swarm  system. 

Figure  3.2  represents  the  entire  model  from  creating  objects  to  the  CID  decision. 
The  main  flow  consists  of  several  sub  models  which  represents  the  sub  phases  and 
actions  occurring  in  each. 
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Figure  3.2  Entire  Simulation  Model 


3.2.1  The  ROI 

The  ROI  is  modeled  as  the  area  bounded  by  a  half-circle  of  radius  50km.  The  area 
within  this  ROI  is  populated  by  stationary  non- threatening  enemies  and  threats.  The 
locations  (xx,  yy)  of  targets  are  initially  unknown.  As  discussed  previously,  there  is  one 
target  placed  in  the  ROI  for  each  MTI.  Targets  are  placed  by  randomly  selecting  a 
distance  between  10km  and  50km  form  the  center  of  the  ROI  and  then  randomly 
selecting  an  angle  between  zero  and  pi  radians.  Figure  3.3  shows  a  notional  ROI  with  an 
example  target  placed. 
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Figure  3.3  Notional  ROI  with  a  target 


Threats  have  anti-craft  capabilities,  such  as  surface-to-air  missiles  (SAMs)  in  each 
cell.  The  threat  is  capable  of  destroying  TUAVs  with  a  kill  probability  by  a  random  draw 
(UNIF  (0,  1)).  The  detail  will  be  explained  later.  Figure  3.3  illustrates  attributes  assigned 
for  each  MTI  to  position  objects  within  the  ROI  before  proceeding  to  the  next  sub  model. 


Figure  3.4  Object  Allocations  for  Environment 
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3.2.2  JSTARs  to  GCS 


Once  a  ROI  is  established,  as  discussed  above,  each  MTI  proceeds  to  be 
developed  through  a  TUAV  tasking  to  obtain  information  on  its  location  which  is  routed 
to  a  GCS.  The  important  logic  in  this  phase  involves  computing  the  amount  of  time  (7)) 
for  a  TUAV  moving  from  one  position  to  the  place  where  the  /'lh  object  is  located. 

Let  (xx,  yy)  be  a  set  of  x  and  v  coordinates  where  an  on-board  sensor  begins  a  scan 
to  discriminate  a  target.  These  coordinates  are  defined  to  be  three  kilometers  (fixed)  away 
from  the  location  of  a  target  passed  as  an  MTI.  The  three  kilometers  (a  survivable 
standoff  range)  represents  a  safe  distance  that  a  TUAV  has  to  keep  from  a  target  to 
prevent  loss  of  TUAVs.  Consider 

XX  —  CLCL  X  Aj  +  (1  —  Aj)  X  CCj 

yy  =  bb  x  +  (1  —  Aj)  x  ddt  (3.1) 

where  Aj  is  defined  as: 

3  (km) 

A;  = - ,  (A  £  R  and  0  <  A  <  1) 

50 (km)  x  y](cc  —  aa)2  +  ( dd  —  bb)2 

and  (aa,  bb)  is  a  set  of  x  and  y  coordinate  representing  a  current  position  of  a  TUAV  and 
(CCj,  ddt)  is  a  set  of  x  and  y  coordinate  implying  a  location  of  the  /th  target. 

Therefore,  a  travel  time  (T()  of  a  TUAV  for  discriminating  the  7th  object  can  be  expressed 
as: 

Ti  —  J(xx  —  aa)2  +  (yy  —  bb)2  x  50 (km)  4-  2(^m/mjn)  (3.2) 

The  distance  from  (aa,  bb)  to  (xx,  yy)  in  our  ROI  is  multiplied  by  50km  to  compute  a  real 
distance,  then  it  is  divided  by  2km/min  implying  an  average  flight  speed  of  a  TUAV  to 
calculate  the  time  (7)). 
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The  Decision  time  is  the  amount  of  time  it  takes  our  ATR  system  to  review 
information  from  a  sensor  on  board  and  make  a  declaration  for  an  MTI.  For  our  single 
TUAV  system  the  decision  time  is  uniformly  distributed  from  one  to  five  minutes,  while 
for  our  swarm  system  with  two  TUAVs  active,  we  use  the  joint  distributions  (1  +  4  x 
BETA(1.03,  1.29)  min  for  system  and  2,  0.59  +  GAMM(0.713,  2.85)min  for  system  3) 
developed  using  another  Arena  model.  To  come  up  with  this  joint  distribution,  two 
hundred  decision  times  were  first  simulated  and  collected  and  then  processed  using  the 
Arena  input  analyzer  to  fit  a  theoretical  distribution  function  to  the  data.  Figure  3.4 
illustrates  the  sub  model  for  collecting  decision  times  to  compute  the  joint  distribution. 


Figure  3.5  Sub  model  for  collecting  decision  times 


For  example,  if  both  TUAVs  declare  a  target  is  present  in  the  simulated  ROI,  the 
declarations  are  saved  in  the  “id.label”  as  an  attribute  and  the  event  is  defined  as  Type  4 
(used  for  routing  later  in  model  and  statistic  collection).  Then  we  store  the  minimum  of 
the  two  TUAV  times  for  making  a  target  declaration.  This  follows  the  logic  for  our  fusion 
rule  where  we  only  require  one  TUAV  declaration  to  make  a  decision.  The  minimum 
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time  in  this  case  is  considered  the  decision  time  for  our  swarm  system.  On  the  other  hand, 
if  two  TUAVs  declare  a  target  is  not  present,  the  maximum  time  for  a  declaration  with 
each  TUAV  is  used.  The  maximum  time  in  this  case  is  considered  the  decision  time  of 
our  swarmed  system.  We  use  the  various  decision  times  as  described  in  coming  up  with 
our  joint  distribution. 

3.2.3  TUAV-IN 

Once  an  MTI  item  is  allocated  a  TUAV  (or  TUAV  swarm),  a  TUAV  moves  to  the 
place  where  the  target  is  located.  It  takes  the  amount  of  time  specified  in  the  “TUAV  to 
GCS”  sub  model.  A  TUAV  first  attempts  to  execute  its  mission  while  remaining  within 
its  survivable  standoff  range.  However,  a  TUAV  might  be  restricted  in  its  execution  due 
to  dust,  cloud,  camouflage,  and  etc.  To  deal  with  this  situation  in  our  model,  we  require  a 
TUAV  (or  TUAV  swarm)  to  proceed  beyond  its  standoff  range  to  perform  discrimination 
a  small  percentage  of  the  time.  Based  on  [21]  we  use  a  fixed  probability  of  92.5%  for 
successful  discrimination  and  transmission  of  target  infonnation  back  to  the  GCS  outside 
of  a  3km  standoff  range.  If  a  TUAV  is  unable  to  perform  discrimination  outside  the 
standoff  range  (7.5%  of  the  time).  The  TUAV  moves  into  the  standoff  range  and  toward 
the  MTI  location  a  random  distance  (UNIF(0.5,  l)km)  which  is  expressed  as  “extra 
moving  distance”  in  Equation  3.4.  While  within  the  standoff  range,  a  TUAV  has  a  50% 
probability  of  being  shot  down  by  enemy  air  defense  systems.  If  shot  down,  another 
TUAV  is  prepared  and  launched  to  continue  the  mission.  Otherwise  the  TUAV  continues 
the  CID  process  by  making  a  declaration  and  continues  back  to  the  L/R  point  or  to 
another  MTI  location.  Similar  logic  is  used  for  our  swarm  system,  with  separate  random 
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draws  taken  to  determine  if  one  or  both  TUAVs  can  perform  their  mission  outside  the 
standoff  range  and  separate  draws  for  survivability  if  one  or  both  are  required  to  move 
into  the  standoff  range. 

When  a  TUAV  is  needed  to  perform  inside  of  the  standoff  range,  an  additional 
travel  time  ( T{ )  of  a  TUAV  for  discriminating  an  7th  object  can  be  calculated  as  follow: 

Let  (xx,  yy )  be  a  set  of  x  and  y  coordinate  that  an  on-board  sensor  of  a  TUAV  begins 
scanning  to  discriminate  a  target  after  additional  moving. 

xx  =  aa  x  A  +  (1  —  A)  x  cc 

yy  =  bb  x  A  +  (1  —  A)  x  dd  (3.3) 

where  (aa,  bb)  is  a  set  of  x  and  v  coordinate  representing  a  current  position  of  a  TUAV 
(i.e.,  the  position  that  a  TUAV  is  supposed  to  execute  discrimination  before  deciding  to 
move  toward  a  target  more)  and  (cc,  dd)  is  a  set  of  x  and  y  coordinate  implying  a  location 
of  a  target,  the  A  is  defined  as, 

3  (km)  —  extra  movinq  distance  (km) 

A  =  — - - -  =  (A  e  R  and  0  <  A  <  1) 

50 (km)  x  y  (cc  —  aa)2  +  (dd  —  bb)2 

Therefore,  an  additional  travel  time  is  calculated  as, 

T[  =  y](xx  -  aa)2  +  (yy  -  bb)2  x  50 (km)  4-  2 &m/min)  (3.4) 

To  decide  success  or  failure  of  the  discrimination,  our  simulation  compares  a 

prior  TPR  (shown  in  Table  3. 1  as  PTP)  in  CM  (or  a  posterior  TPR  in  case  of  TUAV 

swarm)  with  a  random  number  representing  the  probability  of  an  object  being 

discriminated  correctly.  For  example,  if  a  posterior  TPR  of  a  TUAV  swann  is  0.9  and  the 

random  number  is  0.46,  the  system  correctly  declares  the  object  a  target.  However,  the 

system  fails  to  discriminate  the  target  if  the  random  number  is  greater  than  the  TPR. 
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Table  3.1  summarizes  the  probability  estimates  associated  with  horizontal  analysis  of 
each  row,  and  the  vertical  analysis  metrics  in  terms  of  the  confusion  matrix  cells,  CM 
(row,  col).  For  our  system  we  are  only  interested  in  the  upper  left  cell  of  this  table  since 
we  only  model  a  single  class  (all  objects  are  targets)  and  are  interested  only  in  the  PTP  (in 
our  case  computed  as  all  objects  declared  targets  divided  by  all  objects). 


Table  3.1  Typical  Performance  Measures  Associated  with  the  Confusion  Matrix  Cells, 

CM  (row,  col)  [11] 

Classifier  “Labels” 


“Target”  1  declaration 

“Friend"  1  declaration 

“ Non-declaration  ” 

p  _  CM  (1.1) 

CM  (1,2) 

™  CM(l  1)  +  CM  (1,2) 

P  =1  -P 

1  FN  1  1  TP 

_  CM  (1,3) 

TF  CM  (1,1)  +  CM  (1,2) 

m  CM  (1,4) 

p  _  CM  (2,1) 

CM  (2,2) 

p  _  CM  (2,3) 

UF  CM  (2  A) 

"  CM  (2,1)  +  CM  (2, 2) 

™  ~  CM  (2,1)+ CM  (2, 2) 

P  =  1  -P 

lTN  1  1  FP 

P  P 

p  -  r  rFP 

P  P 

f  -  rTrFN 

Prej  =  ^T^UT  +  ^F^VF 

CR  P  P  +P  P 

1  F  M  FP  —  1  T 1  TP 

NC  P  P  +  P  P 

1  F  1  TN  '  1  T  1  FN 

p  =\~p 

‘Dec  1  ‘  REJ 

3.2.3. 1  Fusion  Process  for  Computing  the  Posterior  TPR 

A  TUAV  swarm  makes  a  CID  decision  through  use  of  a  Boolean  fusion  rule 
(logical  OR).  When  one  of  UAVs  in  a  swarm  (or  both)  declares  an  enemy  is  present,  the 
swarm  system  finally  returns  a  label  of  enemy.  Figure  3.4  presents  the  Arena  model 
developed  with  this  fusion  logic  used  to  generate  the  TPR  for  our  swann  system. 


38 


Figure  3.6  Fusion  Process  in  Discrimination 


The  Boolean  logical  OR  rule  is  used  to  combine  the  output  labels  from  our  two 
TUAV  sensors.  This  rule  returns  a  target  for  the  simulated  system  if  either  or  both 
TUAVs  conclude  an  object  under  investigation  is  a  target.  With  two  sensors  fused,  2=4 
different  sensor  labels  are  generated  for  any  given  assessment  of  a  target.  For  example, 
the  following  labels  of  the  fusion  can  be  possible:  both  TUAV  1  and  TUAV  2  declare  a 
target  is  present  (defined  as  case  4  in  the  simulation  model  for  routing),  TUAV  1  declares 
a  target  is  present  while  TUAV  2  does  not  (defined  as  case  6),  TUAV  2  declares  a  target 
is  present  while  TUAV  1  does  not  (defined  as  case  5),  or  both  TUAVs  declare  an  target  is 
not  present  (defined  as  case  7).  Therefore,  successful  declaration  of  a  target  where  at  least 
one  of  TUAVs  correctly  discriminates  a  target  can  be  represented  by  4,  5,  and  6  as  the 
output  fusion  process.  If  it  is  labeled  as  7,  the  discrimination  process  has  failed  for  that 
object. 
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3.2.4  TUAV-OUT 


After  completing  its  mission,  a  TUAV  (or  TUAVs)  has  three  options.  One  is  to 
return  its  L/R  site  and  to  wait  for  the  arrival  of  a  fresh  MTI  (i.e.,  it  does  not  land  but  flies 
over  the  site)  when  there  is  no  MTI  item  waiting  to  be  developed.  However,  if  a  fresh 
MTI  item  arrives  at  a  TUAV  GCS  before  a  TUAV  reaches  the  L/R  site,  a  TUAV  is 
redirected  from  the  current  position  (defined  as  (aa,  bb)  in  our  map)  to  the  new  ROI 
(defined  as  (cc,  dd))  and  perfonns  a  new  task.  If  there  are  any  MTIs  waiting  to  be 
developed  after  processing  the  current  MTI,  a  TUAV  is  immediately  redirected  to 
execute  new  mission.  We  use  a  FIFO  rule  here  to  deal  with  which  MTI  item  is  first 
served  among  MTIs  waiting  to  be  developed.  If  there  are  any  undeveloped  MTIs,  a 
TUAV  heads  to  the  first  one  received. 

3.3  Variables  and  attributes 

In  this  section  the  attributes  and  variables  used  throughout  the  model  and  in 
collecting  statistics  are  listed  and  explained  briefly  in  Table  3.2.  These  attributes  and 
variables  are  used  as  part  of  the  logic  throughout  the  simulation  model 
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Table  3.2  Variables  and  Attributes  in  the  model 


Dist. 

Distance  from  a  L/R  site  of  a  TUAV  to  the  position  of  a  TUAV 
beginning  its  discrimination  process 

Degree 

Centering  around  a  L/R  site,  an  angle  of  the  position  of  a  TUAV 
beginning  its  discrimination  process 

Prob .  swarm,  shot,  down 

Probability  of  either  or  both  TUAVs  in  a  swarm  being  shot  down 
by  enemy  air  defense  systems  while  executing  a  mission  within  its 
survivable  standoff  range 

Prob.TUAV  1  .shot. down 

Probability  of  TUAV  1  being  shot  down  while  executing  a 
mission  alone  within  its  survivable  standoff  range 

Prob.TUAV  1  .shot.down 

Probability  of  TUAV2  being  shot  down  while  executing  a 
mission  alone  within  its  survivable  standoff  range 

Prob .  discriminated . 
correctly 

Probability  of  an  object  being  discriminated  correctly 

1  Prob .  Obj  ect.  discriminated 
.inside 

Probability  of  an  object  being  discriminated  by  TUAV  1  inside 
the  standoff  range 

2Prob .  Obj  ect .  discriminated 
.inside 

Probability  of  an  object  being  discriminated  by  TUAV2  inside 
the  standoff  range 

Decide,  time,  swarm 

Decision  time  (min)  of  a  swarm  system  for  declaring  an  object 

Decide.time.TUAV  1 

Decision  time  (min)  of  TUAV1  for  declaring  an  object, 

Decide.time.TUAV2 

Decision  time  (min)  of  TUAV2  for  declaring  an  object, 

TPR.of.swarm 

True  Positive  Rate  of  a  swarm  system  when  two  TUAVs 
perfonn  discrimination 

TPR.of.TUAVl 

True  Positive  Rate  of  TUAV  1  when  TUAV1  performs 
discrimination  alone  in  a  swarm  system 

TPR.of.TUAV2 

True  Positive  Rate  of  TUAV2  when  TUAV1  performs 
discrimination  alone  in  a  swarm  system 

Prep.time.TUAV  1 

Time  to  prepare  and  launch  TUAV  1 

Prep  .time.  TU A  V 2 

Time  to  prepare  and  launch  TUAV2 

More.travel.TUAV  1 

Additional  distance  in  our  map  that  TUAV  1  has  to  move  to 
discriminate  an  object  inside  the  standoff  range 

More.travel.TUAV2 

Additional  distance  in  our  map  that  TUAV2  has  to  move  to 
discriminate  an  object  inside  the  standoff  range 

aa 

x-coordinatc  of  the  current  position  of  a  TUAV  in  a  single 

TUAV  system  or  a  leader  TUAV  in  a  swann  system 

bb 

y-coordinate  of  the  current  position  of  a  TUAV 

Num.of.TUAVs 

Number  of  TUAVs  in  the  simulation 

Num.TUAVs.  shot,  down 

Number  of  TUAVs  shot  down 

Total,  decision.time 

Total  decision  time  of  a  system  until  the  simulation  is  terminated 

Total .  TU  A  V  s .  Prep .  time 

Total  time  that  a  GCS  spend  to  prepare  and  launch  TUAVs 

id.  label 

Decision  label  of  a  TUAV  in  discrimination  process. 
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3.4  Verification  and  Validation 


In  this  section,  verification  and  validation  of  the  simulation  model  are  discussed. 
Verification  is  a  process  that  is  used  to  check  if  a  simulation  model  is  coded  correctly  or 
logic  in  the  model  is  correct.  Validation  is  the  process  of  establishing  a  high  level  of 
assurance  that  a  simulation  model  matches  the  real  system. 

The  simulation  model  can  be  verified  by  the  animation  feature  of  Arena. 
Whenever  a  module  for  building  each  process  of  CID  is  added,  our  simulation  model  is 
run  to  check  if  the  flow  of  entity  is  appropriate  or  not. 

Currently,  no  UAV  swarm  concept  is  being  employed  based  on  the  finding  in  this 
region.  Therefore,  the  face  validity  is  used  for  validation  of  our  simulation  model.  Our 
results  of  model  are  reasonable  with  expected  performances  for  the  hypothesized  system 

3.8.  Conclusion 

This  chapter  focused  on  description  and  assumptions  of  the  simulation  model, 
concepts  of  building  a  simulation  model  and  application  of  steps  for  making  our 
simulation  model  reasonable.  The  specific  variables  and  attributes  for  our  research  were 
also  described.  Results  and  analysis  from  our  model  are  discussed  in  the  next  chapter. 


42 


IV.  Analysis  and  Results 


General 

This  chapter  focuses  on  model  results  and  analysis.  In  the  first  section,  we  present 
the  factors  used  and  output  data  obtained  in  our  model,  and  the  appropriate  replication 
length  and  number  also  are  discussed.  The  following  include  comparisons  between 
different  competing  systems  on  the  various  performance  measures  and  the  analysis  of 
responses  from  this  simulation  model. 

4.1  Measure  of  Effectiveness 

Measures  of  Effectiveness  (MOE)  in  various  fields  of  military  require  some  kind 
of  quantification.  Number  of  targets  detected,  survivability  score,  aircraft  availability  and 
accident  rates,  and  many  other  measures  serve  to  reduce  a  large  number  of  data  into 
meaningful  information.  Quantitative  methods  are  used  to  analyze  and  explain  actions  or 
outputs  to  a  decision  maker  such  as  the  commander  of  a  Brigade. 

MOEs  should  be  selected  and  interpreted  with  care,  because  MOEs  that 
appropriately  distill  and  precisely  reflect  reality  help  decision  makers  make  informed, 
timely  decisions.  If  measures  are  poorly  selected  or  wrongly  interpreted,  the  collection 
and  analysis  of  the  MOE  may  waste  time  and  effort  [25], 

In  this  research,  the  outputs  we  captured  are  the  accumulated  accuracy,  the 
number  of  TUAVs  shot  down,  mean  of  simulated  mission  time,  mean  of  decision  time 
per  a  MTI,  and  total  time  for  TUAV  preparation. 
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Each  scenario  is  run  for  30  replications.  This  number  of  replications  was  selected 
because  30  replications  are  generally  enough  to  meet  Central  Limit  Theorem  condition 
for  independent  and  identically-distributed  random  variables  and  give  us  a  reasonable 
half  width  (for  each  MOE  plus  or  minus  5%). 

4.2  Simulation  Results 

In  this  section  we  present  the  simulation  results  of  the  systems  using  the  following 
three  different  configurations. 

•  A  high-value  sensor  which  has  a  high-level  TPR  (called  system  1) 

•  Two  low-value  sensors  which  has  a  low-level  TPR  (system  2) 

•  Two  middle-value  sensors  which  has  a  middle-level  TPR  (system  3) 


Table  4. 1  Specification  of  a  sensor  used  in  each  system 


Level 

TPRs 

Decision  time 

Prob.  of  a  TUAV  successfully  discriminating 
target  inside  of  the  stand-off  range 

High 

UNIF(0. 85,0.97) 
Improved  by  30% 

UNIF(0.91,4.91) 
Improved  by  3% 

0.938875 

Improved  by  1.5% 

Improvement 
based  on 
the  value  of  low- 
level 

Middle 

UNIF(0.71,0.83) 
Improved  by  1 0% 

UNIF(0.97,4.97) 
Improved  by  1  % 

0.929625 

Improved  by  0.5% 

Low 

UNIF(0.64,0.76) 

UNIF(1,5) 

0.925 
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Figure  4.1  Accuracy  of  the  systems  as  a  function  of  No.  of  MTIs 


To  assess  the  performances  of  the  system  using  one  of  three  configurations 
described  above,  we  simulated  each  system  in  the  environments  with  20,  40,  60,  80,  and 
100  targets  (labeled  as  MTIs  in  our  figures).  For  all  the  simulation  runs  in  this  paper,  the 
homogeneous  targets  are  randomly  assigned  to  the  ROIs  while  a  system  executes  its 
mission.  We  synchronize  use  of  the  same  random  number  streams  for  the  same  purpose 
when  simulating  each  of  our  systems,  therefore,  the  location  of  targets  and  the  threats 
level  of  targets  are  all  the  same  for  each  configuration. 

From  Figure  4. 1  we  can  see  that  the  System  3  out-performs  the  others  in  terms  of 
the  accumulated  accuracy  of  a  system.  The  perfonnance  of  System  3  with  two  middle- 
value  sensors  fares  better  than  the  system  with  two  low-value  sensors  and  the  system  one 
high-value  sensor.  Although  the  TPR  of  the  sensor  used  in  System  3  is  less  than  that  of 
the  sensor  in  System  1,  the  perfonnance  of  System  3  is  more  successful  than  that  of 
System  1  because  of  the  increased  TPR  with  fusion  of  information  in  our  swarm  system. 
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We  looked  at  additional  simulation  outputs  to  study  the  performances  of  these 
systems  for  the  same  ROIs.  Figure  4.2  shows  loss  of  TUAVs  and  preparation  time  for 
fresh  TUAVs  over  the  same  range  of  targets. 


Figure  4.2  Number  of  TUAVs  shot  down  and  TUAV  preparation  time 
as  a  function  of  No.  of  MTIs 


These  values  give  a  good  indication  of  the  survival  rate  of  a  single  TUAV 
(System  1)  versus  use  of  a  swarm  system  (System  2  and  3).  It  is  obvious  that  our 
cooperative  method  provides  a  significant  improvement  in  the  number  of  TUAVs  shot 
down.  As  discussed  before,  a  TUAV  can  be  shot  down  when  it  performs  discrimination 
inside  of  the  standoff  range.  If  it  executes  its  mission  outside  of  the  range,  the  TUAV  is 
secure  from  any  threats.  In  addition,  if  at  least  one  TUAV  in  the  swarm  can  execute  its 
task  outside  this  range,  our  cooperative  algorithm  does  not  allow  the  remaining  TUAV  to 
move  inside  of  the  range  for  discrimination.  It  only  allows  TUAVs  to  move  inside  the 
standoff  range  when  both  TUAVs  fail  to  perform  discrimination  outside  of  the  range. 
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This  is  why  the  numbers  of  TUAVs  shot  down  with  one  of  our  swann  systems  are 
significantly  less  than  that  number  for  System  1,  the  single  TUAV  system.  The  pattern  of 
the  two  plots  in  Figure  4.2  looks  similar  to  each  other.  It  does  make  sense  because  the 
smaller  loss  of  TUAVs,  the  less  preparation  time  for  fresh  TUAVs,  however,  there  is 
little  difference  between  the  performances  of  the  two  swarm  systems  for  either  plot  in 
Figure  4.2.  This  indicates  that  this  MOE  is  more  dependent  on  our  cooperative  algorithm 
with  a  swann  system  than  the  increased  TPR  of  System  3  over  System  2. 


Figure  4.3  Mean  of  Decision  and  Simulated  mission  time  as  a  function  of  No.  of  MTIs 


The  left  plot  in  Figure  4.3  shows  mean  of  decision  time  representing  the  mean 
time  that  a  system  spends  collecting  data  and  making  a  declaration  for  an  object.  The 
right  plot  shows  mean  of  simulated  mission  time.  The  performances  of  the  proposed 
fusion  algorithm  (System  2  and  3)  were  compared  to  that  of  the  system  using  one  TUAV. 
In  our  fusion  algorithm,  successful  declaration  of  a  target  is  defined  as  the  case  where  at 
least  one  TUAV  correctly  discriminates  a  target.  We  also  modeled  the  single  TUAV 
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decision  time  as  being  slightly  improved  over  the  two  swann  systems.  Therefore,  we 
expected  to  see  some  decrease  in  the  swann  system  performance  for  this  measure.  As  we 
can  see  from  Figure  4.3,  the  performances  of  the  swann  systems  are  slightly  better  than 
that  of  the  single  TUAV  system.  This  is  due  to  once  again  to  our  fusion  rule.  Once  one  of 
the  TUAVs  declares  a  target  is  present,  the  other  TUAV  does  not  spend  time  on  the  same 
object  leading  to  a  decrease  in  decision  time.  In  addition  since  we  assumed  the  middle- 
level  sensor  used  in  the  System  3  is  slightly  improved  based  on  the  low-level,  the 
performance  of  System  3  is  better  than  that  of  System  2. 

Our  mean  of  simulated  mission  time  (Figure  4.3)  depends  strongly  on  TUAVs 
preparation  time,  travel  time,  inter-arrival  time  between  MTIs,  and  decision  time.  The 
simulated  mission  times  of  the  swann  systems  are  less  than  that  of  the  System  1 .  As 
presented  before,  the  total  TUAVs  preparation  time  and  decision  time  of  the  swarm 
system  are  less  than  those  of  the  System  1 .  In  addition,  since  the  locations  of  targets  and 
the  flight  speed  of  TUAVs  in  our  simulation  are  ah  the  same  while  the  TUAVs  of  each 
system  perform  its  mission,  the  total  travel  time  of  each  system  differs  little  from  each 
other.  This  is  why  the  swarm  systems  show  better  performance  in  terms  of  shorter  mean 
of  simulated  mission  time.  As  the  number  of  MTIs  increase,  mean  of  simulated  mission 
time  of  three  systems  tends  to  decrease.  At  the  beginning  of  these  simulations,  an  inter¬ 
arrival  time  between  MTIs  strongly  affects  the  simulated  mission  time  because  the  longer 
an  inter-arrival  time  between  MTIs,  the  bigger  the  simulated  mission  time.  However,  as 
the  number  of  MTIs  increase  more  of  the  time  there  will  be  MTIs  waiting  to  be 
developed.  In  such  a  case  the  inter-arrival  time  does  not  affect  the  simulated  mission  time 
anymore  and  the  travel  times  of  TUAVs  decrease  because  TUAVs  directly  move  to  the 
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next  MTI  without  spending  time  going  back  and  forth  to  L/R  point.  This  is  why  the  mean 
of  simulated  mission  time  decreases  as  the  number  of  MTIs  increase. 

4.3  Comparison  of  Competing  Systems 

In  this  section  statistical  analysis  is  performed  to  answer  the  question  of  whether 
or  not  these  measures  represent  a  statistically  significant  difference  in  performance  of  the 
three  systems.  One  approach  is  to  compare  the  confidence  intervals  for  a  pair  of  means 
and  unless  they  overlap  we  can  say  the  difference  between  these  system  means  is 
statistically  significant.  This  is  an  informal  test  and  we  set  up  a  hypothesis  to  provide  us  a 
decision  based  on  a  chosen  confidence  level  in  order  to  quantify  how  certain  we  are  of 
the  difference  [2:164]. 

The  paired  t-test  is  a  common  method  to  compare  alternatives  of  real  systems. 

The  statistic  for  this  test  is  the  difference  between  the  two  alternatives  for  each 
replication.  The  null  hypothesis  is  that  the  mean  of  the  differences  is  zero.  This  approach 
does  not  require  equal  variances  and  independence  between  systems,  but  the  sample  size 
should  be  equal. 

Because  the  output  from  a  simulation  is  a  random  variable,  the  precision  of  the 
results  from  the  simulation  will  be  decided  by  the  variance  of  the  output.  One  statistical 
technique  to  reduce  such  a  variance  is  the  use  of  Common  Random  Numbers  (CRN) 
along  with  the  paired  t-test.  The  idea  of  using  CRN  is  to  ensure  that  alternative 
configurations  of  a  model  are  different  only  due  to  those  configurations  and  not  due  to 
different  random  conditions  used  in  the  model,  so  the  use  of  an  exclusive  random  number 
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stream  for  each  place  in  the  simulation  where  a  random  variate  draw  occurs  is  advised 
[2:169]. 

Three  different  systems  are  used  in  this  study.  All  requirements,  such  as  dedicated 
random  number  streams  and  starting  replications  for  each  system  with  the  same  random 
number,  related  to  getting  the  most  form  using  CRN  are  incorporated  in  our  simulation. 
These  different  system  configurations  are  examined  in  terms  of  accuracy,  number  of 
TUAVs  shot  down,  TUAV  preparation  time,  decision  time,  and  simulated  mission  time 
of  the  systems.  The  following  discussion  presents  paired  t-test  results  comparing 
performance  of  the  single  TUAV  and  the  swann  systems  for  sets  of  20,  60,  and  100 
targets  (MTIs).  All  confidence  intervals  used  are  two-tailed  at  a  95%  level  of  confidence 
and  all  p-values  are  two-tailed. 
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The  first  paired  t-test  (Table  4.2)  was  performed  to  determine  if  the  means  of  the 


difference  between  these  systems  is  zero  in  terms  of  accuracy. 


Table  4.2  Paired  t-test  for  Accuracy  (%) 


MTIs=20 

o 

Mean 

St.  Dev. 

SE  Mean 

Sys  1 

ETfl 

0.90833 

0.07081 

0.01293 

Sys  3 

0.93167 

0.06757 

0.01234 

Diff. 

Era 

-0.02333 

0.05529 

0.01010 

95%  Cl 

-0.04398  -0.00269 

P-value 

0.02812 

MTIs=20 

o 

Mean 

St.  Dev. 

SE  Mean 

Sys  1 

Era 

0.90833 

0.07081 

0.01293 

Sys  2 

0.88000 

0.07944 

0.01450 

Diff. 

Era 

0.02833 

0.05972 

0.01090 

95%  Cl 

0.00603  0.05063 

P-value 

0.01456 

MTIs=60 

o 

Mean 

St.  Dev. 

SE  Mean 

Sys  1 

Era 

0.90556 

0.04160 

0.00759 

Sys  3 

0.93111 

0.03178 

0.00580 

Diff. 

-0.02556 

0.03681 

0.00672 

95%  Cl 

-0.03930  -0.01181 

P-value 

0.00068 

MTIs=60 

□ 

Mean 

St.  Dev. 

SE  Mean 

Sys  1 

Era 

0.90556 

0.04160 

0.00759 

Sys  2 

0.88389 

0.04844 

0.00884 

Diff. 

0.02069 

0.04424 

0.00822 

95%  Cl 

0.00386  0.03752 

P-value 

0.01120 

MTIs=100 

o 

Mean 

St.  Dev. 

SE  Mean 

Sys  1 

Era 

0.91033 

0.02977 

0.00543 

Sys  3 

0.92600 

0.02513 

0.00459 

Diff. 

-0.01567 

0.02837 

0.00518 

95%  Cl 

-0.02626  -0.00507 

P-value 

0.00517 

MTIs=100 

o 

Mean 

St.  Dev. 

SE  Mean 

Sys  1 

Era 

0.91033 

0.02977 

0.00543 

Sys  2 

0.88700 

0.03583 

0.00654 

Diff. 

0.02333 

0.03871 

0.00707 

95%  Cl 

0.00888  0.03779 

P-value 

0.00256 

In  the  case  of  the  comparison  of  System  1  (single  TUAV)  and  System  2,  the 
means  of  the  difference  are  significantly  greater  than  zero,  none  of  the  confidence 
intervals  contain  zero,  and  all  the  P-values  are  less  than  0.05,  providing  evidence  that 
System  1  shows  better  performance  than  System  2.  On  the  other  hand,  for  System  1  and 
System  3  all  the  means  of  differences  are  less  than  zero,  none  of  the  confidence  intervals 
contain  zero,  and  all  the  P-values  are  less  than  0.05.  It  indicates  that  the  accuracy  of 
System  3  is  higher  than  that  of  the  System  1 .  However,  the  differences  range  from  only 
about  one  to  four  percent  which  may  not  be  practically  significant,  but  would  need  to  be 
researched  further. 
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The  second  paired  t-test  (Table  4.3)  was  used  to  test  the  null  hypothesis  that  the 
numbers  of  TUAVs  shot  down  with  the  single  TUAV  and  each  swarm  system  are 
statistically  the  same.  Here  all  confidence  intervals  lie  above  zero  indicating  that  the 
number  of  TUAVs  shot  down  with  System  1  is  statistically  larger  either  swarm  system. 
The  results  indicate  that  in  terms  of  survival  rate  the  perfonnance  of  a  swarm  system  is 
better  than  that  of  a  single  TUAV  system. 


Table  4.3  Paired  t-test  for  Number  of  TUAVs  shot  down 


1  No.  of  TUAVs  shot  down  ( 

MTIs=20 

N 

Mean 

St.  Dev. 

SE  Mean 

Sys  1 

30 

1.30000 

1 .20773 

0.22050 

Sys  3 

30 

0.23333 

0.67891 

0.12395 

Diff. 

30 

1 .06667 

1 .25762 

0.22961 

95%  Cl 

0.59706  1.53627 

P-value 

0.00007 

|  No.  of  TUAVs  shot  down  | 

MTIs=20 

N 

Mean 

St.  Dev. 

SE  Mean 

Sys  1 

30 

1.30000 

1.20773 

0.22050 

Sys  2 

30 

0.23333 

0.67891 

0.12395 

Diff. 

30 

1.06667 

1.25762 

0.22961 

95%  Cl 

0.59706  1.53627 

P-value 

0.00007 

|  No.  of  TUAVs  shot  down  I 

MTIs=60 

N 

Mean 

St.  Dev. 

SE  Mean 

Sys  1 

30 

4.00000 

2.01717 

0.36828 

Sys  2 

30 

0.43333 

0.81720 

0.14920 

Diff. 

30 

3.56667 

2.22344 

0.40594 

95%  Cl 

2.73642  4.39691 

P-value 

0.00000 

1  No.  of  TUAVs  shot  down  I 

MTIs=60 

N 

Mean 

St.  Dev. 

SE  Mean 

Sys  1 

30 

4.00000 

2.01717 

0.36828 

Sys  3 

30 

0.43333 

0.81720 

0.14920 

Diff. 

30 

3.56667 

2.22344 

0.40594 

95%  Cl 

2.73642  4.39691 

P-value 

0.00000 

1  No.  of  TUAVs  shot  down  1 

MTIs=100 

N 

Mean 

St.  Dev. 

SE  Mean 

Sys  1 

30 

6.16667 

2.54725 

0.46506 

Sys  3 

30 

0.70000 

1.31700 

0.24045 

Diff. 

30 

5.46667 

2.51524 

0.45922 

95%  Cl 

4.52746  6.40587 

P-value 

0.00000 

|  No.  of  TUAVs  shot  down  I 

MTIs=100 

N 

Mean 

St.  Dev. 

SE  Mean 

Sys  1 

30 

6.16667 

2.54725 

0.46506 

Sys  2 

30 

0.70000 

1.31700 

0.24045 

Diff. 

30 

5.46667 

2.51524 

0.45922 

95%  Cl 

4.52746  6.40587 

P-value 

0.00000 
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The  third  paired  t-test  (Table  4.4)  was  performed  to  test  if  there  is  difference  in 
the  perfonnance  representing  the  TUAV  preparation  time  of  the  swarm  systems  and  the 
single  TUAV  system.  As  shown  in  this  table,  none  of  the  confidence  intervals  contain 
zero  and  all  the  P-values  are  below  0.05,  so  the  difference  is  statistically  significant. 
Because  the  smaller  value  in  this  measure  implies  the  better  performance,  we  can  say  that 
the  swarm  systems  lead  to  a  decrease  in  TUAV  preparation  time. 


Table  4.4  Paired  t-test  for  TUAV  preparation  time  (minutes) 


|  TUAVs  Prep,  time  | 

MTIs=20 

m 

Mean 

St.  Dev. 

SE  Mean 

Sys  1 

RTil 

15.46600 

20.39687 

3.72394 

Sys  2 

4.71390 

13.97316 

2.55114 

Diff. 

10.75210 

23.06731 

4.21150 

95%  Cl 

2.13862  19.36558 

P-value 

0.01620 

|  TUAVs  Prep,  time  | 

MTIs=20 

o 

Mean 

St.  Dev. 

SE  Mean 

Sys  1 

Bm 

15.46600 

20.39687 

3.72394 

Sys  3 

4.71390 

13.97316 

2.55114 

Diff. 

10.75210 

23.06731 

4.21150 

95%  Cl 

2.13862  19.36558 

P-value 

0.01620 

|  TUAVs  Prep,  time  | 

MTIs=60 

□ 

Mean 

St.  Dev. 

SE  Mean 

Sys  1 

ED 

42.78557 

34.08991 

6.22394 

Sys  2 

8.81667 

16.98095 

3.10028 

Diff. 

33.96890 

38.99293 

7.11910 

95%  Cl 

19.40870  48.52910 

P-value 

0.00005 

1  TUAVs  Prep,  time  | 

MTIs=60 

o 

Mean 

St.  Dev. 

SE  Mean 

Sys  1 

ED 

42.78557 

34.08991 

6.22394 

Sys  3 

8.81667 

16.98095 

3.10028 

Diff. 

33.96890 

38.99293 

7.11910 

95%  Cl 

19.40870  48.52910 

P-value 

0.00005 

S  TUAVs  Prep,  time  j 

MTIs=100 

o 

Mean 

St.  Dev. 

SE  Mean 

Sys  1 

64.85667 

36.05568 

6.58284 

Sys  3 

14.96887 

28.01269 

5.11439 

Diff. 

49.88780 

38.27825 

6.98862 

95%  Cl 

35.59447  64.18113 

P-value 

0.00000 

|  TUAVs  Prep,  time  j 

MTIs=100 

m 

Mean 

St.  Dev. 

SE  Mean 

Sys  1 

64.85667 

36.05568 

6.58284 

Sys  2 

14.96887 

28.01269 

5.11439 

Diff. 

49.88780 

38.27825 

6.98862 

95%  Cl 

35.59447  64.18113 

P-value 

0.00000 

The  fourth  set  of  paired  t-tests  examining  the  difference  of  mean  of  decision  time 
between  the  single  and  swann  system  are  shown  in  Table  4.5.  For  the  smallest  number  of 
targets  (MTIs  =  20),  there  is  no  statistical  difference  in  the  mean  of  decision  time 
between  the  systems  because  the  confidence  intervals  contain  zero  and  the  P-values  are 
much  larger  than  0.05.  However,  as  the  number  of  targets  increase  (which  clearly 
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increase  mission  time),  the  difference  between  systems  grows,  with  a  significant 
difference  between  System  1  and  3  at  60  targets,  and  significant  difference  between  the 
single  TUAV  and  both  swarm  systems  at  100  targets. 


Table  4.5  Paired  t-test  for  mean  of  decision  time  (minutes) 


|  MTIs=20 

m 

Mean 

St.  Dev. 

SE  Mean 

ESI 

ETfl 

2.899352 

0.297227 

0.055194 

■II 

2.734633 

0.324942 

0.059326 

1  Diff. 

ED 

0.16287 

0.474003 

0.086541 

95%  Cl 

-0.01413  0.33987 

P-value 

0.06991 

|  MTIs=20  | 

d 

Mean 

St.  Dev. 

SE  Mean 

mmmm 

El 

2.899352 

0.297227 

0.055194 

HE9 

2.827463 

0.228567 

0.04173 

!  Diff.  I 

0.07004 

0.375886 

0.068627 

95%  Cl 

-0.07032  0.21040 

P-value 

0.31589 

|  MTIs=60 

□ 

Mean 

St.  Dev. 

SE  Mean 

EH 

El 

2.919843 

0.139549 

0.025478 

■sdsfll 

2.737937 

0.175466 

0.032036 

I  Diff. 

El 

0.181907 

0.208019 

0.037979 

95%  Cl 

0.10423  0.25958 

P-value 

0.00005 

|  MTIs=60  | 

d 

Mean 

St.  Dev. 

SE  Mean 

EH 

El 

2.919843 

0.139549 

0.025478 

KH 

2.883583 

0.151786 

0.027712 

I  Diff.  | 

0.03626 

0.203542 

0.037161 

95%  Cl 

-0.03974  0.11226 

P-value 

0.33726 

|  MTIs=100 

d 

Mean 

St.  Dev. 

SE  Mean 

EH 

El 

2.914603 

0.107031 

0.019541 

EH 

2.710083 

0.115408 

0.02107 

I  Diff. 

0.20452 

0.151809 

0.027716 

95%  Cl 

0.14783  0.26121 

P-value 

0.00000 

|  MTIs=1 00 1 

d 

Mean 

St.  Dev. 

SE  Mean 

BSE 

El 

2.914603 

0.107031 

0.019541 

EH 

2.854067 

0.112013 

0.020451 

i  Diff.  j 

0.060537 

0.16211 

0.029597 

95%  Cl 

0.00000  0.12107 

P-value 

0.04999 

Finally,  the  null  hypothesis  that  the  mean  of  the  difference  in  simulated  mission 
time  of  the  swarm  system  and  single  TUAV  system  is  zero  is  tested  by  a  paired  t-test. 
These  results  are  shown  in  Table  4.6.  The  difference  between  System  1  and  System  2  is 
not  statistically  significant  for  simulated  mission  time,  while  the  difference  between 
System  1  and  System  3  is  statistically  significant.  However,  it  is  found  that  the  difference 
between  System  1  and  System  3  may  not  be  practically  significant,  because  the 
performance  of  System  3  has  decreased  by  about  6%  compared  to  the  performance  of 
System  1  and  the  difference  can  be  considered  to  be  not  significant  in  the  real  world. 
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Table  4.6  Paired  t-test  for  mean  of  simulated  mission  time  (minutes) 


MTIs=20 

m 

Mean 

St.  Dev. 

SE  Mean 

Sys  1 

ESI 

20.64983 

3.209447 

0.585962 

BEE 

19.4644 

2.24118 

0.409182 

i  Diff.  1 

1.185433 

3.913194 

0.714448 

95%  Cl 

-0.27578  2.64664 

P-value 

0.10785 

MTIs=20 

m 

Mean 

St.  Dev. 

SE  Mean 

Sys  1 

El 

20.64983 

3.209447 

0.585962 

EH 

19.34167 

2.697191 

0.492437 

i  Diff. 

El 

1.308167 

3.088029 

0.563794 

95%  Cl 

0.15508  2.46126 

P-value 

0.02756 

|  MTIs=60 

n 

Mean 

St.  Dev. 

SE  Mean 

El 

18.97897 

1.869383 

0.341301 

BSE 

17.9007 

1 .345255 

0.245609 

I  Diff. 

El 

1 .078267 

1 .676573 

0.306099 

95%  Cl 

0.45222  1.70431 

P-value 

0.00144 

|  MTIs=60  | 

m 

Mean 

St.  Dev. 

SE  Mean 

BEE 

El 

18.97897 

1.869383 

0.341301 

B3E 

18.1579 

1.475109 

0.269317 

i  Diff.  j 

0.821067 

2.326918 

0.424835 

95%  Cl 

-0.04782  1.68995 

P-value 

0.06310 

MTIs=100 

d 

Mean 

St.  Dev. 

SE  Mean 

Sys  1 

El 

18.3621 

1.251194 

0.228436 

Sys  2 

17.78377 

1.095421 

0.199996 

Diff. 

0.578333 

1.615134 

0.294882 

95%  Cl 

-0.02477  1.18143 

P-value 

0.05952 

MTIs=100 

□ 

Mean 

St.  Dev. 

SE  Mean 

Sys  1 

El 

18.3621 

1.251194 

0.228436 

Sys  3 

17.5714 

1 .09267 

0.199493 

Diff. 

El 

0.7907 

1 .43563 

0.262109 

95%  Cl 

0.25463  1.32677 

P-value 

0.00527 

4.4  Conclusion 

In  this  chapter,  statistical  analysis  on  the  outputs  from  our  simulation  was 
discussed.  First  the  proper  length  and  number  of  replications  of  our  simulation  model  was 
determined,  then  the  factors  which  contribute  to  our  model  were  presented.  Finally  by 
using  a  paired  t-test,  the  perfonnances  of  the  systems  having  different  configuration  were 
analyzed.  The  following  chapter  will  discuss  the  conclusions  and  recommendations. 
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V.  Conclusions  and  Recommendations 


General 

This  chapter  gives  a  short  summary  and  conclusions  of  this  research,  and  then 
recommendations  for  future  research  are  discussed. 

5.1  Summary  and  Conclusions 

The  objective  of  this  research  was  to  analyze  a  TUAV  swarm  with  low-value 
sensors  versus  a  single  UAV  with  a  high-value  sensor  to  investigate  the  effects  of 
changing  defined  variables  in  the  CID  process.  First,  we  constructed  a  detailed  computer 
model  of  the  entire  CID  process  with  only  targets  present  after  making  several 
assumptions,  and  defining  a  number  of  variables,  and  desired  statistics.  We  defined  three 
different  systems  (a  single  TUAV  and  two  different  swarm  configurations)  and 
performed  30  replications  for  each.  The  outputs  from  these  different  configurations  were 
captured  and  five  perfonnance  measures  were  evaluated. 

In  this  study,  we  simulated  a  CID  process  using  a  TUAV  swarm  for 
discriminating  a  target  within  an  unknown  region.  A  key  issue  to  capture  swarm  behavior 
of  TUAVs  is  to  design  logic  such  that  the  swarm  of  TUAVs  perfonns  the  discrimination 
mission  cooperatively  based  on  information  they  get.  We  developed  a  cooperative  fusion 
algorithm  using  a  Boolean  OR  rule. 

Five  measures  are  evaluated  for  the  comparison  of  different  systems.  These 
measures  are  accuracy,  number  of  TUAV  shot  down,  TUAV  preparation  time,  mean  of 
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decision  time,  and  mean  of  simulated  mission  time.  The  performance  achieved  using  a 
single  TUAV  was  compared  with  the  swarm  systems.  As  shown  in  Table  5.1,  the  swann 
systems  are  consistently  better  (across  all  performance  measures).  For  System  2  we 
picked  the  sensor  capabilities  for  the  swann  to  roughly  match  the  TPR  performance  of 
the  single  TUAV.  For  System  3  the  sensor  levels  were  selected  in  order  for  the  TPR 
performance  of  System  3  to  show  a  modest  improvement  over  System  2. 


Table  5.1  System  showing  better  performance  in  each  environment 


No.  of 
MTIs 

No.  of 
TUAVs 
shot  down 

TUAV 
preparation 
time  (min) 

Decision 

time 

(min) 

Simluation 

time 

(min) 

Accuracy 
(%)  ' 

20 

Swarm 

Swarm 

No  diff. 

No  diff. 

Swarm 
(System  3) 

60 

Swarm 

Swarm 

Swarm 
(System  3) 

Swarm 

Swarm 
(System  3) 

100 

Swarm 

Swarm 

Swarm 

Swarm 

Swarm 
(System  3) 

The  results  of  the  statistical  tests  indicate  we  can  cut  down  our  expenses  for 
purchasing  air  frames  and  sensors  by  employing  the  swarm  system  that  we  proposed 
because  our  swarm  system  shows  much  better  performance  than  the  single  TUAV  system 
in  terms  of  number  of  TUAVs  shot  down.  Besides,  we  can  decrease  research  and 
development  expenditures  for  developing  sensor  capabilities  because  as  we  can  see  in 
Table  5.1,  sending  two  coordinated  TUAVs  (our  swann  systems)  with  less  complex 
sensors  leads  to  an  equivalent  or  even  better  CID  call  than  sending  a  single  TUAV  with 
more  sophisticated  sensor.  Cost  is  a  driving  factor  for  constructing  and  maintaining  a 
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military  system  and  a  decision  maker  wants  to  achieve  the  maximum  efficiency  at  a 
minimum  of  effort  and  cost.  Because  of  the  significant  reduction  in  costs  with  our  swarm 
system  from  fewer  TUAV  platform  lost  to  enemy  fire  in  the  scenarios  examined  along 
with  a  much  lower  projected  sensor  cost,  we  conclude  that  our  swann  system  with  fused 
sensors  presents  the  potential  for  future  employment. 

5.2  Recommendations  for  Future  Study 

This  simulation  model  can  be  improved  by  adding  an  additional  process  such  as 
classification  and  identification  as  well  as  including  both  friendly  objects  and  clutter  in 
each  ROI.  For  people  who  study  ATRs,  an  entire  CID  process  consists  of  five  processes. 
These  are  detection,  discrimination,  classification,  recognition,  and  identification.  By 
modeling  from  detection  to  identification  with  different  object  types,  we  can  get  more 
robust  measures  of  system  performance  in  a  more  realistic  operational  environment. 

Current  sensor  and  classifier  fusion  texts  provide  evidence  not  only  to  Boolean 
fusion  rules’  general  use  and  acceptance  but  also  to  their  easy  implementation.  However, 
as  mentioned  in  Chapter  3,  the  use  of  Boolean  fusion  for  decision  labels  results  in 
suboptimal  for  more  than  two  sensors  when  each  sensor  is  optimized  independently. 
Modeling  different  levels  of  correlation  among  fused  sensors  and  exploring  other  fusion 
rules  may  lead  to  improved  system  performance. 

In  addition,  by  adding  and  modeling  costs  for  operating  the  system  (i.e.,  research 
and  development  expenditures  and  expenses  for  purchasing  sensors  and  air  frames,  we 
can  get  more  robust  measures  related  to  cost-effectiveness. 
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Appendix  A.  Decision  times  of  System  2  for  computing 
the  Joint  Distribution  of  Decision  time 


Obser. 

Decision 

Time 
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Appendix  C.  Decision  times  of  System  3  for  computing 
the  Joint  Distribution  of  Decision  time 
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